Commit Graph

208 Commits

Author SHA1 Message Date
Ivan Grokhotkov
13881a3832
idf.py: ensure that build log is always sanitized from color sequences
The actual output from the build tool (CMake/Ninja) may or may not
contain color escape codes, depending on various factors. The output
written to the log file should never include color escape codes,
though. This is because color escape codes in files are usually not
rendered as "color" in editors, and complicate reading. Also escape
codes would break the regular expressions used to display hints for
compilation errors.
2022-08-30 16:38:14 +02:00
Ivan Grokhotkov
d04b554065
tools: idf.py: enable CLICOLOR_FORCE for interactive builds
If stdout is a TTY (meaning that the output is not redirected), tell
the build tool (GNU Make or Ninja) to enable colorized output.

GNU Make and Ninja also check if their stdout is redirected and
strip color escape sequences in that case. CLICOLOR_FORCE environment
variable overrides this behavior.

With this change, if the compiler was launched with the
-fcolor-diagnostics flag and idf.py output is not redirected, the
final output in the terminal will be colorized.

(-fcolor-diagnostics is handled at CMake level by the previous commit)
2022-08-30 16:38:13 +02:00
Roland Dobai
cb81d14892 Merge branch 'bugfix/build_color_output_windows' into 'master'
tools: idf.py: use actual sys.stdout instead of the memoized one

See merge request espressif/esp-idf!19847
2022-08-30 16:31:42 +08:00
Ivan Grokhotkov
b548d9345e Revert "Merge branch 'feature/build_color_output' into 'master'"
This reverts merge request !19330
2022-08-30 15:14:47 +08:00
Ivan Grokhotkov
2c5b62f10e
tools: idf.py: use actual sys.stdout instead of the memoized one
This fixes the issue with build output not being colorized on Windows,
while the hints messages are colorized.

The issue occurred because sys.stdout and sys.stderr get overridden
by colorama.init() at runtime, but the default argument
output_stream=sys.stdout holds the reference to the"original"
sys.stdout.

colorama.init() (which, by the way, gets called via a curious chain
of imports, via idf_component_tools.manifest and tqdm package)
overrides standard streams, on Windows only. The overridden streams
contain logic to convert ANSI color codes into Windows Console API
calls to colorize the text.

Since read_and_write_stream function used the default value of
output_stream evaluated at module loading time, it was using the
original sys.stdout, not the one overridden by colorama.

One extra note is that while this does fix the coloring issue, the
solution is a bit fragile, as it relies on one of the following
(on Windows):
- colorama.init() is called (this can change if idf-component-manager
  stops importing tqdm)
- Sufficiently new version of Windows 10 is used, and ANSI color codes
  support is enabled in the Registry.
2022-08-30 01:30:32 +02:00
Roland Dobai
84acc58d3c Merge branch 'fix/disable_idf_py_moinitor_hints' into 'master'
Tools: Disable idf.py hints for IDF Monitor

Closes IDFGH-8110

See merge request espressif/esp-idf!19782
2022-08-29 22:51:55 +08:00
Ivan Grokhotkov
712743d777
idf.py: ensure that build log is always sanitized from color sequences
The actual output from the build tool (CMake/Ninja) may or may not
contain color escape codes, depending on various factors. The output
written to the log file should never include color escape codes,
though. This is because color escape codes in files are usually not
rendered as "color" in editors, and complicate reading. Also escape
codes would break the regular expressions used to display hints for
compilation errors.
2022-08-29 11:15:16 +02:00
Ivan Grokhotkov
12abe1f316
tools: idf.py: enable CLICOLOR_FORCE for interactive builds
If stdout is a TTY (meaning that the output is not redirected), tell
the build tool (GNU Make or Ninja) to enable colorized output.

GNU Make and Ninja also check if their stdout is redirected and
strip color escape sequences in that case. CLICOLOR_FORCE environment
variable overrides this behavior.

With this change, if the compiler was launched with the
-fcolor-diagnostics flag and idf.py output is not redirected, the
final output in the terminal will be colorized.

(-fcolor-diagnostics is handled at CMake level by the previous commit)
2022-08-29 11:15:00 +02:00
Roland Dobai
94a2d9f16a Merge branch 'bugfix/fix_hints_templates' into 'master'
tools: fix hints templates

See merge request espressif/esp-idf!19789
2022-08-29 16:43:14 +08:00
Roland Dobai
55b7b42724 Merge branch 'bugfix/fix_CN_letters_in_monitor' into 'master'
tools: fix multi-byte character appearance in idf.py monitor

See merge request espressif/esp-idf!19566
2022-08-29 16:36:01 +08:00
Djordje Nedic
5ee663d592 tools: Add CSV support to idf_size.py
This adds CSV support to idf_size.py and idf.py size actions and using the --format argument which accepts 'text', 'json' or 'csv' as input.

idf_size.py --json argument is deprecated but left to avoid a breaking change.

For idf.py size actions OUTPUT_JSON environment variable set at configuration time is overriden at target build time if --format is used.

Additionally, this commit refactors big parts of code, unified usage of json_dict and manually generated dictionaries for textual output and improves code quality in many parts.
2022-08-26 15:34:21 +02:00
simon.chupin
e635492c95 tools: fix hints templates 2022-08-25 18:00:23 +02:00
Roland Dobai
ff38cb8e0d Tools: Disable idf.py hints for IDF Monitor
The feature will be re-enabled later after a proper fix for the
following Github issue.

Closes https://github.com/espressif/esp-idf/issues/9610
2022-08-25 14:53:41 +02:00
simon.chupin
1cb4f8fdf4 tools: fix multi-byte character appearance in idf.py monitor 2022-08-23 13:25:38 +00:00
songruojing
304a8f142d esp32c6: introduce the target
Add esp32c6 target to tools and Kconfig
Create directories and files that are essential for `idf.py --preview set-target esp32c6`
2022-08-19 11:13:02 +08:00
simon.chupin
cc1eb81b2e tools: add support of templates for hints 2022-08-18 13:37:32 +02:00
Vilem Zavodny
a00e8771a7 tjpgd: Remove ROM patches and tests. 2022-08-17 14:42:45 +02:00
Mahavir Jain
6c2c05e8a0 Merge branch 'feature/esp_cryptoauthlib_from_manager' into 'master'
Push out esp-cryptoauthlib to component manager

See merge request espressif/esp-idf!19208
2022-08-16 17:01:53 +08:00
Roland Dobai
f8bcec07fa Merge branch 'fix/idf_py_logging' into 'master'
Tools: Fix idf.py hints to be enabled all the time and being able to disable them

See merge request espressif/esp-idf!19542
2022-08-15 22:36:15 +08:00
Roland Dobai
bd5e999408 Merge branch 'fix/idf_hints_errors_in_name' into 'master'
Tools: Correct idf.py hints for possible syntax errors

Closes IDF-5745

See merge request espressif/esp-idf!19545
2022-08-15 22:21:54 +08:00
Roland Dobai
152ce8884b Tools: Fix idf.py hints to be enabled all the time and being able to disable them 2022-08-15 13:01:39 +02:00
Sudeep Mohanty
f6d9617b28 hints: added compilation hints for FreeRTOS breaking changes 2022-08-15 12:48:51 +02:00
Roland Dobai
23441c2102 Tools: Correct idf.py hints for possible syntax errors 2022-08-12 16:50:30 +02:00
Mahavir Jain
42b4601039
tools: add hint for using esp-cryptoauthlib from manager 2022-08-12 11:27:22 +05:30
Armando (Dou Yiwen)
cdd50aff1a Merge branch 'docs/add_migration_notes_about_removing_esp_adc_cal_component' into 'master'
Docs: add migration notes / hints about removing esp adc cal component

See merge request espressif/esp-idf!19392
2022-08-12 10:52:40 +08:00
Roland Dobai
cacd27f93e Merge branch 'staging/add_hints_esp_system' into 'master'
Build: add hints for the breaking changes of esp_system related components

Closes IDF-5757

See merge request espressif/esp-idf!19499
2022-08-11 15:51:19 +08:00
Omar Chebib
49613ab832 Build: add hints for the breaking changes of esp_system related components 2022-08-11 11:35:04 +08:00
Armando
d792b49804 idf.py hints: add build error hints when legacy adc calibration driver is used 2022-08-11 11:26:20 +08:00
Laukik Hase
f887922cf4
protocols/provisioning: Added hints for breaking changes
- For migrating from ESP-IDF v4.4.x to v5.0
2022-08-10 10:39:54 +05:30
Roland Dobai
e2abad6db9 Merge branch 'feature/add-esp-hw-support-breaking-change-hints' into 'master'
system/esp_hw_support: Added hints for breaking changes

Closes IDF-5758

See merge request espressif/esp-idf!19388
2022-08-09 21:40:38 +08:00
Roland Dobai
0b2acbf974 Merge branch 'feature/system_migration_hints' into 'master'
idf.py hints: added hints for misc system breaking changes.

Closes IDF-5759

See merge request espressif/esp-idf!19390
2022-08-09 21:26:58 +08:00
Guillaume Souchere
dfcd7c3b88 esp_hw_support: add hints for breaking changes.
hints are added:
- to indicate that esp_cpu_ccount_t must be replaced by esp_cpu_cycle_count_t
- to indicate that esp_cpu_get/set_ccount must be replaced by esp_cpu_get/set_cycle_count
- to indicate that soc/cpu.h and compare_set.h were removed and user must include esp_cpu.h instead
- to indicate that esp_intr.h was removed and user must include esp_intr_alloc.h instead
- to indicate that esp_panic.h was made private and user must use the functionalities from esp_debug_helper.h instead
- to indicate that spilock.h, clk_ctrl_os.h and rtc_wdt.h must be included without the soc/
- to indicate that soc_log.h was renamed esp_hw_log.h and made private
2022-08-09 07:15:57 +02:00
Roland Dobai
311ab428c7 Merge branch 'bugfix/idf_py_gdb_s3_c3' into 'master'
tools: use built-in USB_SERIAL_JTAG for "idf.py openocd" on C3 and S3

See merge request espressif/esp-idf!19418
2022-08-08 22:53:39 +08:00
Ivan Grokhotkov
d366664b0d
tools: use built-in USB_SERIAL_JTAG for "idf.py openocd" on C3 and S3
Since both chips have built-in JTAG functionality, and there are no
official boards with FT2232H for these chips, use the built-in JTAG
by default.

To use them with esp-prog, set:
  OPENOCD_COMMANDS="-f board/esp32c3-ftdi.cfg"
or pass this via the --openocd-commands argument to idf.py.
2022-08-05 18:22:30 +02:00
Marius Vikhammer
dec246af1f idf.py hints: fix header not found regex to also match headerpaths with path seperator 2022-08-05 11:07:48 +08:00
Marius Vikhammer
5beea010e2 idf.py hints: added hints for misc system breaking changes. 2022-08-04 15:34:15 +08:00
Alexey Lapshin
5fb109f2e8 Tools: Add hint to resolve [u]int32_t formatting errors 2022-08-02 12:45:10 +04:00
simon.chupin
5468d79382 tools: fix idf.py monitor consol problem
Closes https://github.com/espressif/esp-idf/issues/9432
2022-07-27 08:35:30 +00:00
Roland Dobai
6928d1f3ec Tools: Add more hint examples on how to resolve breaking changes of v5.0 2022-07-22 13:47:31 +02:00
Roland Dobai
9012e7b59f Merge branch 'bugfix/fix_NotImplementedError_on_windows_by_updating_error_msg' into 'master'
tools: Improve the error message for handling NotImplementedError on Windows

See merge request espressif/esp-idf!19097
2022-07-20 16:48:40 +08:00
Marek Fiala
9c0969b512 Tools: Export selected target actions
Added option --json to action help, for export selected target actions
2022-07-19 20:02:55 +02:00
simon.chupin
a141be4add tools: Improve the error message for handling NotImplementedError on Windows 2022-07-18 19:03:55 +02:00
simon.chupin
cd10c328e5 tools: fix confserver out of limit error
Closes https://github.com/espressif/esp-idf/issues/9335
2022-07-12 11:27:03 +02:00
Marius Vikhammer
398df3b318 dfu: add esp32s3 to supported DFU targets for idf.py 2022-07-11 15:58:21 +08:00
simon.chupin
2bf6ed4f2b idf.py: Fix IOError that costs CI failure 2022-07-04 08:15:23 +00:00
simon.chupin
43c69f0910 idf.py: Add automated hints on how to resolve errors 2022-07-04 08:15:23 +00:00
Roland Dobai
8dddb8b596 Revert "Merge branch 'feature/clippy' into 'master'"
This reverts merge request !16998
2022-06-29 16:46:47 +08:00
simon.chupin
c6a6eaeb60 idf.py: Add automated hints on how to resolve errors 2022-06-23 14:09:34 +02:00
simon.chupin
44f3c19fa9 tools: Add python types hints 2022-06-15 14:33:29 +02:00
simon.chupin
1c5e596633 idf.py: Change copyright in tools dir 2022-05-24 14:01:50 +02:00
morris
334126315f esp32c2: level up from preview targets 2022-05-12 05:18:57 +00:00
Martin Gano
89f754183e Merge branch 'contrib/github_pr_8788' into 'master'
Add idf.py monitor argument --no-reset (-R) (GitHub PR)

Closes IDFGH-7189, IDFGH-7301, and IDFGH-5963

See merge request espressif/esp-idf!18010
2022-05-10 23:34:37 +08:00
Martin Gaňo
c02c0cc9b7 Tools: Add --no-reset option for IDF Monitor in order to avoid resetting the chip target upon connection
Closes https://github.com/espressif/esp-idf/issues/8889

Closes IDFGH-7189, IDFGH-7301, IDFGH-5963

Closes https://github.com/espressif/esp-idf/issues/7651

Merges https://github.com/espressif/esp-idf/pull/8788
2022-05-10 14:19:37 +02:00
Nonoo
9266a7c52e Add idf.py monitor argument --no-reset (-R) to prevent resetting the MCU target on monitor startup
Add idf.py monitor argument --no-reset (-R) to prevent resetting the CPU on monitor startup

idf.py monitor: fix type signature

idf.py monitor: fix reset key shortcut when --no-reset (-R) argument is used

idf.py monitor: change --no-reset (-R) argument descriptions in help

idf.py monitor: simplify --no-reset (-R) argument checks

idf.py monitor: add warning if --no-reset is used, but --port is not given

idf.py monitor: ignore --no-reset if --port is not given
2022-05-03 14:52:34 +02:00
Roland Dobai
9b64c40981 Merge branch 'feature/add_freertos_gdb_integration' into 'master'
tools: add freertos support to GDB

Closes GCC-226

See merge request espressif/esp-idf!17571
2022-04-28 14:36:58 +08:00
radim.karnis
1cdaf4a831 bugfix: Fix port autodetection with esptool package 2022-04-27 09:34:25 +00:00
Alexey Lapshin
a2f6595c5f tools: add freertos support to GDB
GDB with python support will automatically load freertos-gdb python module.
It comes to GDB with commands starting with 'freertos' to show human-readable
tables with freertos task/queue/timer information

Python module URL: https://pypi.org/project/freertos-gdb
2022-03-25 15:25:36 +04:00
Marek Fiala
b5c374ae3e tools: Deprecated commands removed from idf.py
Remove deprecated commands from idf.py, but they are still kept working as cmake deprecated targets.
2022-03-25 09:08:53 +01:00
Marek Fiala
3f6add5964 tools: set baudrate and port with otatool.
Bugfix: Allow setting options port -p and baudrate -b, with idf.py otatool commands.

Closes https://github.com/espressif/esp-idf/issues/8317
2022-03-11 12:27:58 +01:00
Roland Dobai
6b42029a8a Tools: Rename the monitor's baud argument for consistency and use the global baud if not defined
Closes https://github.com/espressif/esp-idf/issues/8316
2022-02-09 16:47:11 +01:00
laokaiyao
cf049e15ed esp8684: rename target to esp32c2 2022-01-19 11:08:57 +08:00
Roland Dobai
b28d7e6850 Tools: Improve the Python package system
Introduce features into the Python package management system & manage
package versions outside of ESP-IDF repo.
2022-01-17 16:54:36 +01:00
Aleksei Apaseev
e30329ffe2 idf_monitor: Changed the elf file to be optional when running "idf.py monitor" command
This feature allows to start IDF Monitor without (re-)building the project.
2022-01-07 17:36:08 +08:00
Roland Dobai
5118dd7cf3 Tools: Add "idf.py save-defconfig" command to generate sdkconfig.defaults 2021-12-15 17:15:31 +01:00
Roland Dobai
460f3ad7b6 Merge branch 'feat/drop_msys' into 'master'
Tools: Remove MSYS/Mingw support

Closes IDF-3954

See merge request espressif/esp-idf!15862
2021-11-18 10:58:43 +00:00
Fu Hanxi
47fc761299 feat(idf.py): add idf.py clang-check and idf.py clang-html-report 2021-11-16 14:08:59 +08:00
Roland Dobai
74ee9b95f9 Tools: Remove makefile support from "idf.py create-project" 2021-11-11 15:32:36 +01:00
Roland Dobai
a1d0d1ffbe Tools: Remove MSYS/Mingw support
MSYS/Mingw was deprecated since v4.0 and it is removed in v5.0. Please
follow the getting started guide of the documentation to set up a
Windows Command Line or Power Shell based environment.
2021-11-10 17:25:07 +01:00
Cao Sen Miao
7f5c415401 ESP8684: introduce a new target esp8684 2021-11-06 17:33:44 +08:00
Fu Hanxi
42405604af idf.py gdb: autoload prefix_map_gdbinit when exists in project_description.json 2021-10-26 14:20:40 +08:00
Fu Hanxi
bee690aa8d idf.py: add linux target support for idf.py flash and idf.py monitor 2021-10-18 11:50:50 +08:00
Marek Fiala
ff18a96f7d tools: replace _ with - in idf.py
Closes https://github.com/espressif/esp-idf/issues/5126
2021-10-13 17:30:38 +08:00
simon.chupin
844dd5075f Tools: make idf_size work with overflow
Closes https://github.com/espressif/esp-idf/issues/6914
closes https://github.com/espressif/esp-idf/issues/4234
2021-09-16 14:12:46 +02:00
Simon
0b9afa509f tools: Add docs subcommand docs which opens browser with documentation for current version of idf and model of your target
Add docs subcommand with options:
--no-browser - Prints url for current documentation instead of opens browser with documentation
--language - Select documentation language
--starting-page - Choice section of documentation to open
--version - Choice version of esp-idf
--target - Choice model of your target
2021-07-12 10:56:37 +02:00
Shu Chen
75bd02bd46 esp32h2: add some more fixes and TODOs 2021-07-01 20:36:39 +08:00
Shu Chen
2b9e8fed71 esp32h2: add esp32h2 build target
Add esp32h2 support in the following components:
 * Kconfig
 * components/esptool_py
 * components/riscv
 * components/xtensa
 * tools
2021-07-01 19:51:33 +08:00
Roland Dobai
483f51cdd0 tools: Some additions to IDF Monitor's timestamps
Additions to https://github.com/espressif/esp-idf/pull/7021
2021-06-13 12:13:39 +02:00
Anton Maklakov
0ed7af2c4c tools: promote ESP32-S3 from preview to supported target 2021-06-07 12:52:28 +07:00
Roland Dobai
6b75bad2b1 tools: Split up large binaries into smaller chunks in the DFU binary
ROM will erase the region a partition is in as soon as it receives the
first bit of the data that is in the partition. For large partitions it
takes more than 5 seconds to erase which is a hard-coded limit in
dfu-utils.

This splits large binaries and adds them by chunks which should avoid
timing-out during flashing.

Closes https://github.com/espressif/esp-idf/issues/6999
2021-05-19 15:15:50 +02:00
Ivan Grokhotkov
952c9a2d05 Merge branch 'feature/esptool_wrapper' into 'master'
tools: Support external Esptool wrappers

Closes IDFGH-5095

See merge request espressif/esp-idf!13389
2021-05-03 09:32:46 +00:00
Martin Babutzka
6faf4941cc tools: Support external Esptool wrappers
Implements https://github.com/jimparis/esptool-ftdi/issues/3

Closes https://github.com/espressif/esp-idf/pull/6879
2021-04-29 08:27:57 +02:00
Roland Dobai
0102cbf334 tools: Add various fixes for idf.py gdbgui
Closes https://github.com/espressif/esp-idf/issues/5968
2021-04-27 17:34:31 +02:00
Roland Dobai
c425ac7bed tools: Don't use revision argument for IDF Monitor if it is not defined 2021-04-15 19:56:39 +02:00
Martin Gaňo
b77addea2f Decompose idf_monitor.py 2021-04-12 10:47:33 +02:00
Martin Vychodil
7bbe2a235f * fixed generation of .gdbinit file (esp32c3 dbg issue) 2021-03-15 15:52:07 +01:00
Roland Dobai
48488f1683 tools: Use git describe --match for filtering out internal names 2021-02-23 06:55:21 +00:00
Martin Gaňo
179e23b38a Fixed automatic connection in idf_monitor
Closes https://github.com/espressif/esp-idf/issues/6415
2021-02-11 17:35:46 +01:00
Ivan Grokhotkov
e70d534566 tools/idf.py: promote esp32c3 from preview to supported target 2021-01-28 11:12:56 +01:00
Fu Hanxi
0146f258d7 style: format python files with isort and double-quote-string-fixer 2021-01-26 10:49:01 +08:00
Ivan Grokhotkov
bac77b4bf7 Merge branch 'feature/idf-list-ports' into 'master'
idf.py: Use function for port detection from esptool.py in idf.py and idf_monitor.py

Closes IDFGH-3104

See merge request espressif/esp-idf!11359
2021-01-08 21:36:08 +08:00
Angus Gratton
60ec13b122 Merge branch 'feature/make_uf2' into 'master'
tools: Wrap flash binaries into a UF2 file for flashing through USB MSC

See merge request espressif/esp-idf!11587
2021-01-05 07:17:27 +08:00
martin.gano
f83039a0cb Imporove port detection in idf.py
Closes https://github.com/espressif/esp-idf/issues/5125
2020-12-31 14:25:52 +01:00
Marius Vikhammer
7fe16bae25 tools: merge C3 changes into master 2020-12-30 07:04:12 +08:00
Roland Dobai
548ea1bdd5 tools: Wrap flash binaries into a UF2 file for flashing through USB MSC
@mmoskal This commit adds basic support for UF2 into ESP-IDF.
2020-12-29 18:14:47 +01:00
Ivan Grokhotkov
8f3a3181e6 tools: decode esp32c3 stack dumps in idf_monitor.py
This follows a similar approach as UART core dump handling in
idf_monitor. Panic handler message is detected in the output,
collected into a file, and the file is passed to the decoding script.
In this case, the decoding script acts as a tiny GDB server, so we
can ask GDB to perform the backtrace.
2020-12-22 20:02:48 +11:00
Jakob Hasse
52093fa4ef linux: added linux target
* add toolchain file
* add linux to preview targets
* add stub for dfu number in cmake
* excluded unity runner per default
* Added esp_attr.h and esp_partition.h linux stubs
* component.cmake check list for emptyness
* added switch for linux in unity cmake file
* Added Linux host example app
2020-12-14 18:53:14 +08:00
martin.gano
346b002d9a enable autocompletion for command idf.py 2020-11-03 23:27:58 +01:00
Mahavir Jain
20af94ff53 Coredump config option rename throughout IDF 2020-09-30 20:22:27 +05:30
martin.gano
b599f127b5 add new command to idf 2020-09-21 23:38:52 +02:00
David Cermak
3b3409fb20 idf.py: Fixed gdb target to exit cleanly when no openocd watch task
When idf.py gdb starts, it expects openocd was started in the background
and creates a thread to watch for openocd errors. when gdb target exits,
the debug_ext.py aims to cleanup all threads and processes, but fails
with traceback if openocd-watch thread not available, which could happen
if openocd started separately.
2020-09-18 08:30:21 +02:00