Commit Graph

231 Commits

Author SHA1 Message Date
Roland Dobai
6edb9982fd Merge branch 'coredump-info-offset_v5.1' into 'release/v5.1'
Tools: coredump-info - fix non-default partition table offset issues (v5.1)

See merge request espressif/esp-idf!26802
2023-11-16 18:56:55 +08:00
Ivan Grokhotkov
684a0936df Merge branch 'contrib/github_pr_12052_v5.1' into 'release/v5.1'
Two small patches for build system and fatfs (GitHub PR) (v5.1)

See merge request espressif/esp-idf!26431
2023-11-14 17:16:43 +08:00
Erhan Kurubas
14241aeb2e fix(coredump-info): set default gdb timeout as 3 seconds 2023-11-13 15:34:52 +01:00
Peter Dragun
5fe418e733 fix(tools/coredump-info): pass non-default partition table offset to esp-coredump
Closes https://github.com/espressif/esp-idf/issues/11564
2023-10-31 15:57:47 +01:00
Michal Jenikovsky
31b4b0a8d4 idf.py: gdb action incorrectly generated EOL gdbinit scripts
GDB on Windows incorrectly reads EOL in the script files causing 'gdb'
action to fail.

(gdb) source .../build/gdbinit/py_extensions
(gdb) source .../build\gdbinit\symbols
add symbol table from file "...\build\bootloader\bootloader.elf"
.../build\gdbinit\symbols:6: Error in sourced command file:
Undefined command: "".  Try "help".

Forcing line separator to '\n' resolved the issue

Signed-off-by: Michal Jenikovsky <jendo@jmsystems.sk>
2023-10-12 04:48:47 +02:00
Peter Dragun
b83a56512a fix(tools): fix autocomplete for --port option
Closes https://github.com/espressif/esp-idf/issues/7970
2023-09-08 09:25:33 +02:00
Roland Dobai
c8970d8205 Merge branch 'feature/requires_hints_v5.1' into 'release/v5.1'
feat(tools): show hints for component dependencies (v5.1)

See merge request espressif/esp-idf!25386
2023-08-25 14:28:13 +08:00
Ivan Grokhotkov
0a029f4bf4 feat(tools): idf.py: implement hints for component reqs
Implementation of hint module for component dependency. It can provide
hint about missing header directory in component's INCLUDE_DIRS or
about missing component dependency in REQUIRES or PRIV_REQUIRES.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-08-17 07:57:38 +02:00
Ivan Grokhotkov
924857d0b4 feat(tools): idf_monitor: support for loadable hint provider modules
Currently hints are supported based on hints.yml only, which may be
limiting for some use cases. This introduces a generic plugin approach,
which allows to implement hint module that doesn't require entry in hints.yml.
Such module has the full command output available and it is not limited to
a single regex in hints.yml.

Note that regex in hint.yml expects the output concatenated into a single line,
but hint modules are getting the output unchanged.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-08-17 07:57:38 +02:00
Frantisek Hrbata
148b7f9a56 feat(tools): add global build context
Introduces get_build_context() helper, which allows to get build context, e.g.
project description, at places where this info is not available. The
build context is set within ensure_build_directory.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-08-17 07:57:38 +02:00
Frantisek Hrbata
6406620712 fix(debug_ext): CTRL+C handling while waiting on gdb process
idf.py spawns gdb process within a thread and uses Thread.join() to wait
for the gdb process to finish. As CTRL+C(SIGINT) is used by gdb to interrupt the
running program, we catch the SIGINT while waiting on the gdb to finish,
and try Thread.join() again.

With cpython's commit

	commit a22be4943c119fecf5433d999227ff78fc2e5741
	Author: Victor Stinner <vstinner@python.org>
	Date:   Mon Sep 27 14:20:31 2021 +0200

	    bpo-45274: Fix Thread._wait_for_tstate_lock() race condition (GH-28532)

this logic doesn't work anymore, because cpython internally marks the
thread as stopped when join() is interrupted with an exception. IMHO
this is broken in cpython and there is a bug report about this
https://github.com/python/cpython/issues/90882. Problem is that
waiting on a thread to finish is based on acquiring a lock. Meaning
join() is waiting on _tstate_lock. If this wait is interrupted, the
above referenced commit adds a logic that checks if the lock is help,
meaning the thread is done and marks the thread as stopped. But there is
no way to tell if the lock was acquired by us running join() or if it's
held by someone else e.g. still by the thread bootstrap code. Meaning
the thread is still running.

I may be missing something, but I don't see any reason why to spawn gdb
process within a thread. This change removes the thread and spawns gdb
directly. Instead waiting on a thread, we wait on the process to finish,
replacing join() with wait() and avoiding this problem.

Closes https://github.com/espressif/esp-idf/issues/11871

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-07-21 12:37:19 +02:00
wuzhenghui
73e380cc3a fix: fix idf passing wrong chip revision to idf_monitor 2023-07-13 10:46:32 +08:00
Peter Dragun
3fc633e326 fix(tools/hints): add guide to fix error with unsupported asyncio
Closes https://github.com/espressif/esp-idf/issues/11732
2023-07-11 10:53:29 +02:00
Alexey Lapshin
71f1bc4453 tools: fix control characters print if hints enabled
Closes https://github.com/espressif/esp-idf/issues/11351
2023-05-23 18:30:22 +08:00
Frantisek Hrbata
b525e273ce tools: fix hints processing in interactive mode
Currently hints are processed only once the process is finished and
exits with non-error exit code. In interactive mode, e.g. for monitor,
we want to process ouput lines for hints right away.

This adds a simple buffer, which keeps the last line and once EOL is
reached, it is processed for hints.

Since the original hints processing was file based, a new helper
function was added to allow processing hints in string.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-05-17 10:56:39 +02:00
laokaiyao
954a6a2cff esp32h4: removed esp32h4 related codes 2023-04-26 18:53:12 +08:00
Roland Dobai
83a682ca1a Merge branch 'monitor_win_color-v5.1' into 'release/v5.1'
bug(idf_monitor): fix color on windows with hints (v5.1)

See merge request espressif/esp-idf!23346
2023-04-24 14:00:05 +08:00
Roland Dobai
4e73cb77e5 Merge branch 'esptool_extra-v5.1' into 'release/v5.1'
feat(esptool): allow to set force for write_flash (v5.1)

See merge request espressif/esp-idf!23338
2023-04-24 13:44:41 +08:00
Peter Dragun
49718b20a5 bug(idf_monitor): fix color on windows with hints
Closes https://github.com/espressif/esp-idf/issues/9610
2023-04-21 14:42:26 +02:00
Peter Dragun
0ed6610212 feat(esptool): allow to set force for write_flash 2023-04-21 09:36:18 +02:00
liqigan
62467fbca5 tools: Update idf-py hints with Bluedroid HFP AG info 2023-04-19 19:10:00 +08:00
Roland Dobai
05a8d0b58a Merge branch 'bugfix/asyncio_progress_term_width' into 'master'
tools: fix progress output in asyncio.run with zero terminal width

Closes IDF-6646

See merge request espressif/esp-idf!22986
2023-04-06 15:23:39 +08:00
Roland Dobai
775f8e7a49 Merge branch 'bugfix/set_target_with_stale_config' into 'master'
tools: fix set-target if IDF_TARGET env is set and stale sdkconfig exists

Closes IDF-7154

See merge request espressif/esp-idf!22950
2023-03-31 17:31:59 +08:00
Frantisek Hrbata
84deb34035 tools: fix progress output in asyncio.run with zero terminal width
Currently the forced progress in RunTool is trying to fit the output
line into a terminal width, but it doesn't take into an account a
situation when the terminal width is reported as zero. This manifests
when running in docker image with redirected output and can be seen
in the github workflow output for esp-idf-ci-action.

docker run --rm -t my_ubuntu_esp python3 -c 'import os,sys; print(os.get_terminal_size(), sys.stdout.isatty())'
os.terminal_size(columns=238, lines=59) True

vs

docker run --rm -t my_ubuntu_esp python3 -c 'import os,sys; print(os.get_terminal_size(), sys.stdout.isatty())' | tee
os.terminal_size(columns=0, lines=0) True

Since the output is reported as tty and the terminal width as 0, the
fit_text_in_terminal() function returns empty string. I also verified this
by running idf.py build inside a testing docker image.

This fix adjusts the fit_text_in_terminal() function to return original
line if the terminal width is zero.

Also simplify the progress print and use same approach as ninja
https://github.com/ninja-build/ninja/blob/master/src/line_printer.cc#L66

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-03-29 17:12:36 +02:00
Frantisek Hrbata
9b731e5e35 tools: fix set-target if IDF_TARGET env is set and stale sdkconfig exists
Currently if the IDF_TARGET env is set, and old sdkconfig exists with
different target value in CONFIG_IDF_TARGET, the set-target action fails
complaining about the IDF_TARGET env and value in sdkconfig being different.
We should ignore IDF_TARGET value from sdkconfig, because we are
actually setting new target and the old sdkconfig is renamed in cmake.

This can be easily reproduced with

---8<---
$ IDF_TARGET=esp32 idf.py set-target esp32
$ IDF_TARGET=esp32s3 idf.py set-target esp32s3

Project sdkconfig '/home/fhrbata/work/hello_world/sdkconfig' was generated
for target 'esp32s3', but environment variable IDF_TARGET is set to 'esp32'.
Run 'idf.py set-target esp32' to generate new sdkconfig file for target esp32.
---8<---

This also adds test for this use case to test_non_default_target.py.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-03-28 11:18:04 +02:00
Roland Dobai
51772f4fb5 Merge branch 'feature/update_rom_elfs' into 'master'
tools: update esp-rom-elf to version 20230320

See merge request espressif/esp-idf!22838
2023-03-28 17:10:44 +08:00
Ivan Grokhotkov
77541faa14
tools: idf.py: remove esp32c6 and esp32h2 from preview targets list 2023-03-21 20:41:44 +01:00
Alexey Lapshin
12ab3625ca tools: update esp-rom-elf to version 20230320 2023-03-20 19:20:34 +08:00
Frantisek Hrbata
a558ad506e tools: fix custom sdkconfig renaming in set-target
Currently the set-target has sdkconfig file name hardcoded to the
default one and doesn't honor custom config paths or names.
IMHO the only place where we can really now the config file name
is in cmake. But also the config should be really renamed only if
the set-target action is running.

This moves the config file renaming into cmake and it's performed only
when _IDF_PY_SET_TARGET_ACTION env. var. is set to 'ON'. This should
hopefully guarantee that it's really renamed only while set-target is
running.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-03-08 16:45:55 +01:00
Frantisek Hrbata
8e912faad1 tools: add target consistency checks for target specified with -D option
Extend existing target consistency checks for the two following cases.
1. Target does not match currently used toolchain
   $ IDF_TARGET=esp32s2 idf.py reconfigure
   $ idf.py -DIDF_TARGET=esp32c3 build

2. Target is ambiguous, because it's specified also as env. var.
   IDF_TARGET=esp32s3 idf.py set-target esp32c2

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-03-08 16:43:42 +01:00
Frantisek Hrbata
0d859f2786 tools: move target guessing into cmake
The _guess_or_check_idf_target() function has sdkconfig and sdkconfig.defaults
file names hardcoded. Since config file names may be specified with SDKCONFIG
or SDKCONFIG_DEFAULTS cmake vars, directly in CMakeLists.txt or passed in with
the -D cmake option, they are not respected.

Problem is when SDKCONFIG or SDKCONFIG_DEFAULTS is set in
CMakeLists.txt. While idf can detect cmake vars passed through it
to cmake via the -D option, detecting SDKCONFIG and SDKCONFIG_DEFAULTS
vars settings in CMakeLists.txt would require to parse it. This seems
like error prone approach. Also if the vars defined by the -D option
are passed directly to cmake, not via idf, they will not be visible to idf.

It seems reasonable to move the logic into cmake, where we know the correct
SDKCONFIG and SDKCONFIG_DEFAULTS values. So the IDF_TARGET detection/guessing
is moved into targets.cmake, where the IDF_TARGET is actually set. The target
is guessed based on the following precendence.

1) $ENV{IDF_TARGET}
2) IDF_TARGET
3) SDKCONFIG
4) sdkconfig
5) SDKCONFIG_DEFAULTS if non-empty or
   $ENV{SDKCONFIG_DEFAULTS} if non-empty or
   sdkconfig.defaults
6) esp32

All config files referred in $ENV{SDKCONFIG_DEFAULTS} and SDKCONFIG_DEFAULTS
are searched, compared to the current behaviour. First target found in the
above chain is used.

The original _guess_or_check_idf_target() is renamed to _check_idf_target() and
used for the target consistency checks only.

The get_sdkconfig_filename() helper is now used to get the sdkconfig file
for consistency checks. It looks in SDKCONFIG specified with the -D
option and project_description.json.

With this change config full paths are reported in messages, so it's clear
e.g. from which config the target was guessed from or which config has
consistency problem. test_non_default_target.py was adjusted to this
change and also new test for testing the IDF_TARGET guessing was added.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-03-08 16:37:58 +01:00
Frantisek Hrbata
a8a4d7c66d tools: add get_sdkconfig_filename() helper
Get project's current sdkconfig file name. It looks in SDKCONFIG cmake var
defined by the -D option and project_description.json. If not found return
default sdkconfig.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-03-08 16:37:58 +01:00
Frantisek Hrbata
e89f980193 tools: add _parse_cmdl_cmakecache() helper
This parses cmakes cache vars defined on command line with -D options
into dictionary. It allows to simplify the check for new cache entries
and also can be re-used for other checks.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-03-08 16:37:58 +01:00
Alexey Lapshin
0da15a7a19 build: add option COMPILER_DISABLE_GCC12_WARNINGS 2023-02-22 05:33:03 +00:00
Martin Vychodil
c58a84f893 Merge branch 'feature/fatfs_format_api' into 'master'
fatfs: added APIs to format a FAT filesystem

Closes IDF-6046

See merge request espressif/esp-idf!21845
2023-02-16 19:59:31 +08:00
Armando
d59fd3e529 fatfs: deprecate esp_vfs_fat_sdmmc_unmount
This API was deprecated before, but without deprecation warning and
migration doc. This commit added deprecation warning and migration doc
to it.
2023-02-16 14:17:51 +08:00
Erhan Kurubas
69b1d1786c docs: JTAG debugging update for ESP32-H2 2023-02-13 22:49:00 +01:00
Roland Dobai
648b1a41c6 Merge branch 'bugfix/dbg_target_hints' into 'master'
tools: enable hints for debug targets

Closes IDF-5795

See merge request espressif/esp-idf!22198
2023-02-10 17:25:27 +08:00
Frantisek Hrbata
dc580bdc97 tools: add hints for openocd
This moves one hint, which was hardcoded in debug_ext.py to
hints.yml and adds a new one when openocd process does not have
permissions to the USB JTAG/serial device.

Also hint replacing the original 'Please check JTAG connection!'
hardcoded message is added.

Suggested-by: Alexey Lapshin <alexey.lapshin@espressif.com>
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-02-03 14:56:14 +01:00
Peter Dragun
3e3533f918 feat(idf_monitor): move idf_monitor to separate repo 2023-02-03 11:20:15 +01:00
Frantisek Hrbata
4ee00623a3 tools: allow to use hints for debug targets
The debug targets are currently not utilizing hints, because they
are not using RunTool() helper from tools.py to spawn sub-processes.
Adjusting debug targets to use RunTool() would require some significant
changes to debug targets and RunTool() as well. Since debug targets
are already storing their output in logs, we can use these and process
them for hints.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-02-01 15:06:39 +01:00
Roland Dobai
d063e48c41 Merge branch 'feature/add-esp32c6-esp32h2-to-roms_json' into 'master'
tools: add esp32c6, esp32h2 to roms.json

Closes IDF-6734

See merge request espressif/esp-idf!22111
2023-01-27 17:01:29 +08:00
Roland Dobai
fda3cf5607 Merge branch 'bugfix/ctrlc_port_detection' into 'master'
tools: allow to interrupt port detection

Closes IDF-6656

See merge request espressif/esp-idf!22062
2023-01-27 16:51:26 +08:00
Alexey Lapshin
dd503eab24 tools: add esp32c6, esp32h2 to roms.json 2023-01-23 12:30:51 +07:00
Frantisek Hrbata
9d1c62becb tools: fix idf.py gdb memory access before remote target is connected
Currently loading of esp32s3 ROM ELF symbols fails with
"Cannot access memory at address 0x3ff194ad". Let's perform
add-symbol-file after connection to the target.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-01-20 11:02:26 +01:00
Alexey Lapshin
6c674f12fa Merge branch 'fix/idf-gdb-remove-ROM-access-workaround' into 'master'
tools: remove obsolete workaround for reading ROM data from flash

See merge request espressif/esp-idf!21944
2023-01-19 23:58:35 +08:00
Erhan Kurubas
7cb79db71e docs: JTAG debugging update for ESP32C6 2023-01-18 15:17:23 +01:00
Djordje Nedic
17d1e9ae36 bugfix: Fix windows path case sensitivity
This commit fixes an issue where paths on Windows are case insensitive, for instance when setting the build folder its name would be converted to lowercase.

The culprit is our realpath() function, that was calling os.path.normcase() internally, since we are removing that call it makes sense to just remove the function entirely and call os.path.realpath() wherever necessary.

Closes https://github.com/espressif/esp-idf/issues/10282
2023-01-17 20:51:46 +01:00
Frantisek Hrbata
92565a259f tools: allow to interrupt port detection
The main idf.py process has a handler for SIGINT, which actually just
ignores it. The get_default_serial_port() function is called within the
idf.py context to detect port for several tools(monitor,flash) and it's
not possible to terminate it because of this. Let's ignore SIGINT only
while running idf_monitor, which uses it to spawn gdb.

Fixes: c6e3eb0922 ("idf.py.exe changes to handle Ctrl+C in correct way. H..")

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-01-17 13:13:54 +01:00
Cao Sen Miao
94120b82c2 esp32h2: add build test 2023-01-17 10:29:04 +08:00