Commit Graph

4862 Commits

Author SHA1 Message Date
Peter Dragun
b83a56512a fix(tools): fix autocomplete for --port option
Closes https://github.com/espressif/esp-idf/issues/7970
2023-09-08 09:25:33 +02:00
Peter Dragun
23a38f0f6f bug(tools): dependency check catch exception when package not installed 2023-08-31 10:24:06 +02:00
Erhan Kurubas
568c397822 feat(coredump): add panic details to the elf file 2023-08-28 18:48:16 +02:00
Aditya Patwardhan
d3f2e4dab5 Merge branch 'fix/esp_prov_recent_changes_v5.1' into 'release/v5.1'
fix(esp_prov): update devices tuple usage due to API deprecations (v5.1)

See merge request espressif/esp-idf!25536
2023-08-28 14:07:35 +08:00
Roland Dobai
06a4943e41 Merge branch 'fix/bin_names_with_dots_v5.1' into 'release/v5.1'
fix(build_system): Allow dots in the name of an app binary (backport v5.1)

See merge request espressif/esp-idf!25276
2023-08-25 14:42:32 +08:00
Roland Dobai
c8970d8205 Merge branch 'feature/requires_hints_v5.1' into 'release/v5.1'
feat(tools): show hints for component dependencies (v5.1)

See merge request espressif/esp-idf!25386
2023-08-25 14:28:13 +08:00
Roland Dobai
a1be2f3613 Merge branch 'bugfix/sbom_hash_check_v5.1' into 'release/v5.1'
fix(test_submodules.py): don't rely on submodule init (v5.1)

See merge request espressif/esp-idf!25057
2023-08-25 14:21:18 +08:00
Roland Dobai
e7722dc79e Merge branch 'bugfix/debug_ext_ctrl_c_v5.1' into 'release/v5.1'
tools: fix waiting on gdb process when interrupted with CTRL+C (v5.1)

See merge request espressif/esp-idf!24933
2023-08-25 14:20:40 +08:00
harshal.patil
7b961d7734
fix(esp_prov): update devices tuple usage due to API deprecations
- a recent commit fixed the API deprecations,
and the BLEDevice object now needs to be accessed through a tuple.
- thus the changes were made in the code wherein the object was used,
but updating the usage in the condition when de
vname is not None got skipped.
2023-08-24 14:48:25 +05:30
Mahavir Jain
a6b5c93ed5 Merge branch 'esp_prov/update_deprecation_warnings_v5.1' into 'release/v5.1'
fix(esp_prov): Fixed the deprecation APIs warnings (v5.1)

See merge request espressif/esp-idf!25150
2023-08-23 18:18:59 +08:00
Aditya Patwardhan
e088c3766b versions: Update version to 5.1.1 2023-08-21 14:24:40 +05:50
Frantisek Hrbata
3e8b1a90cc ci(tools): fix test_hints.py to run on windows
1. The original test, before hint modules support was added, used
tempfile.NamedTemporaryFile in a way which is not supported on windows.
It was having the file open, which the hints tried to read it, leading
the EPERM exception. The docs[1] says this is not supported.
<quote>
Whether the name can be used to open the file a second time, while the
named temporary file is still open, varies across platforms (it can be
so used on Unix; it cannot on Windows)
</quote>

2. The hint module component_requirements test used the idf.py directly,
which is idf.py.exe on windows.Now it's starting idf.py through python.
We could probably used shell=True, but this approach is used in other
tests too.

Anyway the test are now passing on windows.

[1] https://docs.python.org/3/library/tempfile.html

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-08-17 07:57:39 +02:00
Frantisek Hrbata
41f029d654 ci(tools): add tests for component_requirements.py hints module
Following tests were added.

1. Test for missing header directory in component's INCLUDE_DIRS
2. Test for missing dependency in component's PRIV_REQUIRES
3. Test for missing dependency in component's REQUIRES
4. Test for dependency in PRIV_REQUIRES which should be in REQUIRES

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-08-17 07:57:38 +02:00
Ivan Grokhotkov
0a029f4bf4 feat(tools): idf.py: implement hints for component reqs
Implementation of hint module for component dependency. It can provide
hint about missing header directory in component's INCLUDE_DIRS or
about missing component dependency in REQUIRES or PRIV_REQUIRES.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-08-17 07:57:38 +02:00
Ivan Grokhotkov
924857d0b4 feat(tools): idf_monitor: support for loadable hint provider modules
Currently hints are supported based on hints.yml only, which may be
limiting for some use cases. This introduces a generic plugin approach,
which allows to implement hint module that doesn't require entry in hints.yml.
Such module has the full command output available and it is not limited to
a single regex in hints.yml.

Note that regex in hint.yml expects the output concatenated into a single line,
but hint modules are getting the output unchanged.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-08-17 07:57:38 +02:00
Frantisek Hrbata
148b7f9a56 feat(tools): add global build context
Introduces get_build_context() helper, which allows to get build context, e.g.
project description, at places where this info is not available. The
build context is set within ensure_build_directory.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-08-17 07:57:38 +02:00
Roland Dobai
8f91dfed12 fix(tools): Increase timeout in idf_tools.py Python tests 2023-08-11 09:25:52 +02:00
Roland Dobai
951f3300ed fix(tools): Keep the original Python environment when idf_tools.py tests are run 2023-08-11 09:25:52 +02:00
Roland Dobai
dced82b949 fix(tools): idf_tools.py tests should keep the original constraint file 2023-08-11 09:25:52 +02:00
radim.karnis
e72cf8c908 fix(build_system): Allow dots in the name of an app binary 2023-08-08 15:13:25 +02:00
Peter Dragun
0be661f8bf fix(tools): check_python_version accept dev releases as requirement 2023-08-04 14:06:35 +02:00
harshal.patil
0169ae545d fix(esp_prov): Fixed the deprecation APIs warnings
- Updated the extraction of service_uuids using AdvertisementData
- Replaced the `get_services()` API with the `services` property
2023-08-02 17:22:04 +05:30
Frantisek Hrbata
bc16331ba0 fix(test_submodules.py): don't rely on submodule init
Current version of the test is using "git-submodule foreach", which
requires submodules to be initialized. Non-initialized submodules are
ignored. Our CI is not performing submodule initialization, but instead
it only downloads the submodule content in tools/ci/ci_fetch_submodule.py
from cache and copies it into the submodule path.

Since we already know the submodule path from .gitconfig, we can use it
as argument to git-ls-tree and avoid calling git-submodule at all. This
allows to perform the test even if the submodules are not initialization
and also it makes the code simpler.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-07-28 12:13:32 +02:00
Frantisek Hrbata
6406620712 fix(debug_ext): CTRL+C handling while waiting on gdb process
idf.py spawns gdb process within a thread and uses Thread.join() to wait
for the gdb process to finish. As CTRL+C(SIGINT) is used by gdb to interrupt the
running program, we catch the SIGINT while waiting on the gdb to finish,
and try Thread.join() again.

With cpython's commit

	commit a22be4943c119fecf5433d999227ff78fc2e5741
	Author: Victor Stinner <vstinner@python.org>
	Date:   Mon Sep 27 14:20:31 2021 +0200

	    bpo-45274: Fix Thread._wait_for_tstate_lock() race condition (GH-28532)

this logic doesn't work anymore, because cpython internally marks the
thread as stopped when join() is interrupted with an exception. IMHO
this is broken in cpython and there is a bug report about this
https://github.com/python/cpython/issues/90882. Problem is that
waiting on a thread to finish is based on acquiring a lock. Meaning
join() is waiting on _tstate_lock. If this wait is interrupted, the
above referenced commit adds a logic that checks if the lock is help,
meaning the thread is done and marks the thread as stopped. But there is
no way to tell if the lock was acquired by us running join() or if it's
held by someone else e.g. still by the thread bootstrap code. Meaning
the thread is still running.

I may be missing something, but I don't see any reason why to spawn gdb
process within a thread. This change removes the thread and spawns gdb
directly. Instead waiting on a thread, we wait on the process to finish,
replacing join() with wait() and avoiding this problem.

Closes https://github.com/espressif/esp-idf/issues/11871

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-07-21 12:37:19 +02:00
Chen Yudong
c4377e8dcc ci(esp_prov): remove import future usage 2023-07-19 14:03:12 +08:00
Peter Dragun
77429e3ef7 fix(tools/python_dep_check): replace deprecated pkg_resources with importlib
Closes https://github.com/espressif/esp-idf/issues/11712
2023-07-14 14:16:10 +02:00
wuzhenghui
73e380cc3a fix: fix idf passing wrong chip revision to idf_monitor 2023-07-13 10:46:32 +08:00
Roland Dobai
7d11517a68 Merge branch 'asyncio_error_v5.1' into 'release/v5.1'
fix(tools/hints): add guide to fix error with unsupported asyncio (v5.1)

See merge request espressif/esp-idf!24701
2023-07-12 10:34:23 +08:00
Jiang Jiang Jian
788f1def16 Merge branch 'bugfix/esp_netif_recieve_returns_v5.1' into 'release/v5.1'
esp_netif: Report error if esp_netif_receive() fails (v5.1)

See merge request espressif/esp-idf!24655
2023-07-12 04:07:17 +08:00
Zim Kalinowski
819da74d9f Merge branch 'bugfix/improve-git-describe-handling-v5.1' into 'release/v5.1'
build system: more accurate error information for git_describe (v5.1)

See merge request espressif/esp-idf!24665
2023-07-11 17:07:40 +08:00
Peter Dragun
3fc633e326 fix(tools/hints): add guide to fix error with unsupported asyncio
Closes https://github.com/espressif/esp-idf/issues/11732
2023-07-11 10:53:29 +02:00
Zim Kalinowski
7bce53cc0d build system: fix init.bat crash in windows when folder contains unicode characters 2023-07-10 11:33:01 +08:00
Zim Kalinowski
d636754e70 build system: more accurate error information for git_describe
Closes https://github.com/espressif/esp-idf/issues/9071

See merge request espressif/esp-idf!23891
2023-07-10 11:31:32 +08:00
David Cermak
cf4c39a225 esp_netif: Make esp_netif_receive() return value configurable 2023-07-07 13:56:13 +02:00
jiangguangming
fc7d4488bf esp_rom: fix rom layout issues 2023-07-07 01:02:07 -07:00
Jiang Jiang Jian
29f52acbfe Merge branch 'bugfix/increase_pm_dump_time_field_length_v5.1' into 'release/v5.1'
Power Management: increase pm_dump time field width (backport v5.1)

See merge request espressif/esp-idf!24450
2023-07-04 12:16:38 +08:00
Roland Dobai
5d35a45a53 Merge branch 'feature/extend_proj_desc_json_v5.1' into 'release/v5.1'
extend information in project_description.json (v5.1)

See merge request espressif/esp-idf!24125
2023-06-30 21:17:05 +08:00
Roland Dobai
d4652daec6 Merge branch 'fix/idf_tools_certificate_v5.1' into 'release/v5.1'
Tools: Update the certificate of idf_tools.py (v5.1)

See merge request espressif/esp-idf!23928
2023-06-30 21:16:32 +08:00
Roland Dobai
df7ab11865 Merge branch 'fix/hints-print-control-characters_v5.1' into 'release/v5.1'
tools: fix control characters print if hints enabled (v5.1)

See merge request espressif/esp-idf!23889
2023-06-30 21:15:48 +08:00
Roland Dobai
c8ea97984f Merge branch 'bugfix/removed_working_directory_error_v5.1' into 'release/v5.1'
Tools bugfix: Removed working directory when using idf.py (v5.1)

See merge request espressif/esp-idf!24150
2023-06-30 21:06:34 +08:00
Roland Dobai
b48beba60f Merge branch 'feature/tools_add_ulp_linux_i686_toolchain_v5.1' into 'release/v5.1'
tools: add ULP toolchain for linux-i686 platform (v5.1)

See merge request espressif/esp-idf!24194
2023-06-30 21:05:29 +08:00
Roland Dobai
9ceed8a0c9 Merge branch 'feature/submodules_sbom_v5.1' into 'release/v5.1'
tools: add sbom information for submodules (v5.1)

See merge request espressif/esp-idf!24240
2023-06-30 21:05:02 +08:00
wuzhenghui
171544ded4 bugfix: increase pm_dump time field width
Closes https://github.com/espressif/esp-idf/issues/11704
2023-06-28 14:37:27 +08:00
radim.karnis
8ed82f11ae fix: Compatibility with pyparsing>=3.1.0 2023-06-25 19:59:13 +02:00
Sachin Parekh
2fb799c52e build_template_app: Remove extra backslash 2023-06-23 15:01:53 +05:30
Fu Hanxi
3cad4ca937 ci: ignore pkg_resources deprecation warning 2023-06-21 07:27:15 +08:00
Frantisek Hrbata
18969310e2 tools: do not disable pytest-embedded
As pointed out by Fu Hanxi, the pytest_build_system job is currently
using --parallel-index and --parallel-count, which are provided by
pytest-embedded, so we should not disable it. Moreover to properly
disable pytest-embedded we should use "no:pytest_embedded". Meaning
this probably was not working as indented anyway.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-06-14 12:42:50 +02:00
Frantisek Hrbata
dbff74bee4 tools: add sbom information for submodules
This adds SBOM information for submodules, which are not managed
by Espressif. Meaning there is no fork for them in the espressif
namespace. Other submodules should add sbom.yml manifest file to
the root of their git repository.

The SBOM information for submodules is stored in the .gitmodules file.
Each SBOM related variable has the "sbom-" prefix and the following
variables may be used:

sbom-version:
   submodule version

sbom-cpe:
   CPE record if available in NVD. This will be used by the SBOM
   tool to check for possible submodule vulnerabilities. The
   version in the CPE can be replaced with the "{}" placeholder,
   which will be replaced by the "sbom-version" value from above.

sbom-supplier:
   Person or organization who is providing the submodule.
   It has to start with "Person:" or "Organization:" prefix
   as required by the SPDX-2.2 standard.

sbom-url:
   URL to the project if exists, e.g. github.

sbom-description:
   Project description.

sbom-hash:
   Submodule SHA as recorded in the git-tree. This field is used by
   CI to check that the submodule checkout hash and info in .gitmodules
   are in sync. IOW if submodule is updated and it has SBOM info in
   .gitmodules, the .gitmodules has to be updated too. The test is
   part of this commit. The checkout has of the submodule can be found
   by using "git submodule status".

Example for micro-ecc submodule
---8<---
[submodule "components/bootloader/subproject/components/micro-ecc/micro-ecc"]
	path = components/bootloader/subproject/components/micro-ecc/micro-ecc
	url = ../../kmackay/micro-ecc.git
	sbom-version = 1.0
	sbom-cpe = cpe:2.3🅰️micro-ecc_project:micro-ecc:{}:*:*:*:*:*:*:*
	sbom-supplier = Person: Ken MacKay
	sbom-url = https://github.com/kmackay/micro-ecc
	sbom-description = A small and fast ECDH and ECDSA implementation for 8-bit, 32-bit, and 64-bit processors
	sbom-hash = d037ec89546fad14b5c4d5456c2e23a71e554966
---8<---

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-06-14 12:38:12 +02:00
Anton Maklakov
9538dc203d tools: add ULP toolchain for linux-i686 platform
Closes https://github.com/espressif/esp-idf/issues/11561
2023-06-13 12:50:04 +07:00
Marek Fiala
8a9f12eaa4 Tools bugfix: Removed working directory when using idf.py
Added check for existence of current working directory when using idf.py

Closes https://github.com/espressif/esp-idf/issues/11476
2023-06-09 14:24:29 +02:00