feat(bootloader_support): Make `esp_secure_boot_verify_sbv2_signature_block()` function public
Closes IDFGH-11599
See merge request espressif/esp-idf!27742
The size of partition of type APP should be multiple of 4 KB. Partition
generation tool now make this as a mandatory requirement. This is
minimum flash erase size. If the size of the APP type partition is not
aligned to 4 KB then the last erase operation could go beyond the allocated
partition and hence may fail. This issue would only be observed when the
firmware size grows very close to the allocated partition size, and hence
causing the OTA update to fail.
For already deployed devices on-field with the size of APP partition not
aligned to flash sector boundary, it is best to ensure that firmware
size always remains within the lower 4 KB boundary of the total
allocated space. While migrating to ESP-IDF 5.3 release, partition table
for an existing project can be adjusted accordingly for the build to
succeed.
Found during discussion in https://github.com/espressif/esp-idf/pull/12460
When idf.py coredump-debug is launched with '--core' argument, it
tries to determine the file format (raw, elf, b64). To detect the
'raw' core dump the code checked if the version word matched one of
the known values.
However, the version word also contains the chip ID in the high
half-word, so the check failed for anything other than the ESP32.
The detection of core file format has been moved to esp-coredump
package in version 1.9.0, including the fix for chip ID.
Reported in https://github.com/espressif/esp-idf/issues/10852
Currently sbom manifest is checked only in .gitmodules and
this check is done in pre-commit and also in CI. Meaning it's running
three times(pre-commit before push if user has it enabled, in CI
as there is the pre-commit run again and again with test in CI). Since
esp-idf-sbom contains a full manifest validation support and pre-commit
plugin for it, let's use it. This removes all the current sbom testing
and replaces it with a signle pre-commit plugin which validates all
manifests files(sbom.yml, idf_component.yml, .gitmodules and also
referenced manifests) in repository. Note that this checks all
manifests, not only ones which were modified. The check is reasonably
fast though, so it should not cause any problem. The reason for
validating all manifest files is that we want to make sure that the sbom
information in .gitmodules is updated too and that the hash
recorded in .gitmodules is up-to-date. Meaning submodule update
would not trigger this plugin, because no manifest was changed.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit enables the standad VERSION argument for the project() macro
in ESP-IDF. The VERSION argument is compilant with the requirements of
cmake 3.16. This commit also adds new test cases for verifying the
validity of the version argument.
Merges https://github.com/espressif/esp-idf/pull/12461
Co-authored-by: Sudeep Mohanty <sudeep.mohanty@espressif.com>
On riscv chips accessing cache mapped memory regions over the ibus would
result in an illegal instructions exception triggering faster than the cache
error interrupt/exception.
Added a cache error check in the panic handler, if any cache errors are active
the panic handler will now report a cache error, even if the trigger exception
was a illegal instructions.
This extension allows running programs in QEMU similar to running
them on a real chip:
- 'idf.py qemu' — builds and runs the program in QEMU. User gets
a QEMU instance launched, and can work with it as a normal QEMU
instance.
- 'idf.py qemu monitor' — same, but QEMU is launched in the
background, and idf_monitor runs in the foreground, showing QEMU
output. Compared to only running 'idf.py qemu' this enables, for
example, automatic backtrace decoding.
- 'idf.py qemu gdb' — launches QEMU in the background and opens an
interactive GDB prompt, connecting it to QEMU.
- 'idf.py qemu --gdb monitor' and 'idf.py gdb' in another shell:
launches QEMU in the background, keeps it suspended until GDB is
connected, and opens idf_monitor. GDB can be used in another shell
to debug the application.
IDF FreeRTOS v10.5.1 no longer accepts out of range xCoreID arguments in
"PinnedToCore" task creation functions when building for single-core. This
commit fixes those violations through ESP-IDF.
change(usb_serial_jtag): Add example for usb serial jtag echo
Closes IDFGH-11503, IDFGH-11493, and IDFGH-11478
See merge request espressif/esp-idf!27366
This commit deprecates the "freertos/xtensa_context.h" and "xtensa/xtensa_context.h"
include paths. Users should use "xtensa_context.h" instead.
- Replace legacy include paths
- Removed some unnecessary includes of "xtensa_api.h"
- Add warning to compatibility header
With 8959555cee7e[1] ("setup_git_directory(): add an owner check for the top..")
git added an ownership check of the git directory and refuses to
run any git commands, even parsing the config file, if the git directory
is not owned by the current user. The "fatal: detected dubious ownership in repository"
is reported.
This fixes CVE-2022-24765[2], which allows to compromise user account. On a
multi-user system or e.g. on a shared file system, one user may create a "rogue"
git repository with e.g. core.fsmonitor set to an arbitrary command. Other user
may unwillingly execute this command by running e.g. git-diff or
git-status within the "rogue" git repository, which may be in one of the parent
directories. If e.g. PS1 is set to display information about a git
repository in CWD, as suggested in Git in Bash[3], the user do not need to run
any git command to trigger this, just entering some subdirectory under
this "rogue" git repository is enough, because the git command will be
started transparently through the script used in PS1. The core.fsmonitor
can be set to arbitrary command. It's purpose is to help git to identify changed files
and speed up the scanning for changed files.
rogue
├── .git # owned by user1
└── dir1 # owned by user2
├── dir2 # owned by user2
└── .git # owned by user2
user1 sets core.fsmonitor for git repository in rogue directory
$ git config --add core.fsmonitor "bash -c 'rm -rf \$HOME'"
user2 enters dir1 and runs e.g. git diff and triggers the core.fsmonitor command.
The ownership check may cause problems when running git commands in
ESP-IDF Docker container. For example user may run the container as
root, but the mounted project may be owned by a particular user.
In this case git will refuse to execute any git command within the
"/project" directory, because it's not owned by root. To overcome this,
git allows to set safe.directories, for which the ownership check is
skipped. The security check may be completely disabled by setting
safe.directories to "*". This solution was proposed in PR 12636[4], but
it would allow make it possible to exploit this vulnerability again.
This fix allows user to specify git's safe.directory in IDF_GIT_SAFE_DIR
environmental variable, which may be set during container startup.
The IDF_GIT_SAFE_DIR has same format as PATH and multiple directories can be
specified by using a ":" separator. To entirely disable this git security check
within the container, user may set IDF_GIT_SAFE_DIR='*'. This might be
heplfull in CI.
Closes https://github.com/espressif/esp-idf/pull/12636
[1] - 8959555cee
[2] - https://nvd.nist.gov/vuln/detail/cve-2022-24765
[3] - https://git-scm.com/book/en/v2/Appendix-A%3A-Git-in-Other-Environments-Git-in-Bash
[4] - https://github.com/espressif/esp-idf/pull/12636
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This integrates esp_idf_size.ng, refactored esp-idf-size version, into
esp-idf and enables it by default. The esp_idf_size.ng may be enabled
by using the --ng option, but also via ESP_IDF_SIZE_NG env. variable,
which is used in this integration.
New -l/--legacy option is added, which enforces usage of the old version.
This option can be also set via "ESP_IDF_SIZE_LEGACY" env. variable.
This should allow to easily switch back to old version if there is any
problem.
The new version is used by default for all formats, except for the "json".
Examples:
$ idf.py size # uses refactored version
$ idf.py size --legacy # uses legacy version
$ idf.py size --l # uses legacy version
$ idf.py size --format json # uses legacy version
$ idf.py size --format json2 # uses refactored version
$ export ESP_IDF_SIZE_LEGACY="1" # use legacy version only from now on
ESP_IDF_SIZE_FORCE_TERMINAL, which forces terminal control codes(colors), is also set
when running from idf.py, so the colors are propagated even if stdout
for esp_idf_size.ng is not attached to tty.
The same changes are applied also to the idf_size.py wrapper.
There is an import check if esp_idf_size.ng is available. If not,
we switch into the legacy mode. This should cover situation when the
esp-idf has support for refactored version, but it's not installed.
This should also allow users to bind to a legacy version(<1.0.0) and the
idf.py size and idf_size.py should still work. This also allow us to
restring the version in constraints file if we need to switch back to
legacy version globally.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Common spi functionality for sharing the SPI bus between modules is moved from esp_driver_spi to
a more fitting location in esp_hw_support (shared HW resource control).
This also allows us to decouple the spi_flash driver from esp_driver_spi, removing
esp_driver_spi and esp_ringbuf from G1 builds.
This commit fixes memory leaks in unity tests when the debug option
CONFIG_FREERTOS_USE_LIST_DATA_INTEGRITY_CHECK_BYTES is enabled. The
commit increases the threshold to 1200 bytes from 1024 bytes.
esp-system and esp-hw-support, and the rest of the g1 components, now only depend on
esp_driver_spi and esp_driver_gpio. Removing the rest of the driver components from g1 builds.
* Removed gpio_types.h as it is already available
for Linux in the hal component.
* Moved gpio_num.h from tools/mocks to soc component
and added missing GPIO pin definitions
Closes https://github.com/espressif/esp-idf/issues/12520
fix(esp_wifi): Fix issue of station disconnecting immediately after scanning
Closes WIFIBUG-132 and IDFCI-1847
See merge request espressif/esp-idf!25915
change(freertos): Upgrade ESP-IDF to use FreeRTOS v10.5.1, remove v10.4.3
Closes IDF-8191, IDF-8200, IDF-8201, IDF-8363, and IDF-8364
See merge request espressif/esp-idf!26261
This commit fixes the FreeRTOS CMock component in the following ways:
- Updated include directories to work with vanilla FreeRTOS v10.5.1
`#include "portmacro.h"` style inclusion.
the default one of python won't do any compress
also limit a bit on the file patterns, otherwise the .bin files
generated by cmake would also be uploaded
* All components which won't build (yet) on Linux are excluded.
This enables switching to Linux in an application without
explicitly setting COMPONENTS to main in the main
CMakeLists.txt.
* ESP Timer provides headers for Linux now
* automatically disabling LWIP in Kconfig if it is not available
doc(linux): brought section
"Component Linux/Mock Support Overview" up to date
hal is a G0 component but assert may link in the symbols in the newlib component.
we'd better not use assert in the hal but use the alternative HAL_ASSERT
fix(heap): fix the issue on esp32c3 where retention memory was exhausted prematurely and preventing the CPU from powering down
Closes IDF-8236
See merge request espressif/esp-idf!25962
This commit deprecates the `#include "freertos/task_snapshot.h" include path:
- Adds compatibility header with compile time warning
- Removes hints related to `task_snapshot.h`
- Adds entry to migration guide
Task snapshot API were never intended to be called from user code. This commit
makes task snapshot a private API thus moves `#include "freertos/task_snapshot.h"`
to `#include "esp_private/freertos_debug.h"`.
Task snapshot related Kconfig options have also been hidden.
Note: Added 'freertos_common' mapping to ldgen mapping execptions lists due to
inlining of 'freertos_debug.h' functions.
Catch all tlsf assertion failure and display a hint for out of bound writing since
all assertion failure are triggered when the tlsf metadata structure has been corrupted.
bugfix(nvs_flash) : fixed nvs_set functions behaviour when called sequentially with same key and different data type(s)
Closes IDFGH-9727
See merge request espressif/esp-idf!25581
Currently the component_requirements hint module does not work
as expected if the component list for a project is trimmed down.
With the new "all_component_info" dictionary info in project_description.json,
the module can produce hints even if cmake's COMPONENTS variable is
set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add new "all_component_info" dictionary into the
project_description.json file. It contains information about all
registered components presented in the __COMPONENT_TARGETS list.
Since components in this list are not fully evaluated, because only the first
stage of cmakefiles processing is done, it does not contain the same information
as the "build_component_info" dictionary. The "type", "file" and "sources" variables
are missing.
Most of the properties are already attached to the component target, so
this only adds INCLUDE_DIRS property to the target during the first cmakefiles
processing stage.
The "all_component_info" dict is generated in a separate function, even
though the original function for "build_component_info" could be
adjusted. This introduces a little bit of boilerplate, but keeps it
logically separated and probably easier if we want to extend it in the
future.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
1. move startup_stack attr from soc_memory_type_desc_t to soc_memory_region_t and
remove unused aliased_iram field
2. all of the last level of RAM is retention dma accessible on esp32c3
3. remove esp32c2 and later chips retention dma accessible memory caps
4. allow allocate memory from RTC_RAM with MALLOC_CAP_EXEC cap
If there is a component(child) within a component(parent), like for test_apps, the parent
component may be wrongly identified as source component for the failed include. This may
lead to a false bug report if the parent component has component, which provides the missing
header, in requirements.
Fix this by looking for the longest matching source component directory.
Suggested-by: Ivan Grokhotkov <ivan@espressif.com>
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
If LCP keepalive mechanism is enabled in menuconfig, it's statically
configured on creation of an interface and cannot be changed runtime. In
some cases it's useful to relax LCP criteria during runtime operation,
for example before initiating OTA. This config option allows for
disabling already enabled LCP echo (this setting becomes effective after
reconnecting, i.e. initializing a new session)
Closes https://github.com/espressif/esp-protocols/issues/287
Add a test app to verify the working of the application when
SECURE_SIGNED_ON_UPDATE_NO_SECURE_BOOT is selected in the menuconfig
and the application is not signed
This commit adds beta support for the FreeRTOS v10.5.1 kernel which can be
enabled by enabling the CONFIG_FREERTOS_USE_KERNEL_10_5_1 option.
The following changes have been made:
- Updated freertos/CMakeLists.txt to build v10.5.1 kernel with v10.4.3. ports
- Updated existing Xtensa and RISC-V ports to work with V10.5.1
- Modifications to other ESP-IDF components to work with v10.5.1
- Added some ESP-IDF specific tracing changes to v10.5.1 kernel
- Make CONFIG_FREERTOS_USE_KERNEL_10_5_1 a public option
Note: The beta release is missing some minor fixes, performance improvements,
and features. Using this beta release for production is not recommended.
Closes https://github.com/espressif/esp-idf/issues/7137
Merges https://github.com/espressif/esp-idf/pull/12093
fix(ll): remove FLAG_ATTR macro
Such kind of operator overload will not work because C++ thinks such overload is ambiguous and it still prefer the built-in one which accepts and returns integer. Manually force type conversion seems to be unavoidable.
Add changes to use fast_pbkdf2 as default for PMK calculations.
fast_pbkdf2 is significantly faster than current implementations
for esp chips.
Also removes unnecessary code for pbkdf-sha256 and pbkdf-sha512.
This commit moves/merges the IDF FreeRTOS "PinnedToCore" task creation
functions from tasks.c/task.h to idf_additions.h/freertos_task_c_additions.h.
Also updated FreeRTOS Mock component to provide mocks for "idf_additions.h"
headers for our mock tests.
It corrupts environment on macOS runner because it uses shell, not docker-executor,
and makes all tree read-only for a non-root user.
+ Replace examples with real files in example README.md
Currently idf.py reports just "Please use idf.py only in an ESP-IDF shell environment".
Sometimes it may be useful to know for which module the import failed.
Also the problem does not have to be related to shell environment only, but the
python venv can be corrupted. This adds a little bit more verbose error
message.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Users of the nvs API are likely to have `nvs_handle_t` in all cases, but
not all of them carry around the partition name and namespace name (as
they aren't needed after creating an `nvs_handle_t`).
Allow this common case to use nvs iteration without them tracking
additional data by using the `nvs_handle_t` to locate the namespace
index and the partition name by introducing an alterate to
`nvs_entry_find` called `nvs_entry_find_in_handle`, which operates
similarly except that it is given a `nvs_handle_t` instead of partition
and namespace strings.
This is somewhat more limited than the `nvs_entry_find` API as one
cannot examine all keys in a given partition.
Previously, TLSP deletion callbacks were...
- Stored in a seprate TCB member "pvThreadLocalStoragePointersDelCallback"
- Called separately via multipole prvDeleteTLS() insertions in tasks.c
This commit refactors how TLSP deletion callbacks are stored and called:
- TLSP deletion callbacks are now stored in "pvThreadLocalStoragePointers"
directly. configNUM_THREAD_LOCAL_STORAGE_POINTERS is doubled in size so that
the deletion callbacks are stored in the latter half of the array
- The callbacks are now called via "portCLEAN_UP_TCB()". As such, the
prvDeleteTLS() additions are no longer needed and the function can be removed
- Removed some legacy TLSP tests using the old method of storing the callback
pointers.
This commit reduces the source code diff between IDF FreeRTOS and upstream
vanilla FreeRTOS, in preparation for v10.5.1 upgrade.
Previously, if CONFIG_FREERTOS_ENABLE_STATIC_TASK_CLEAN_UP was enabled, users
would provide a definition for a vPortCleanUpTCB() hook function that is called
right before a task's memory is freed in prvDeleteTCB(). However,
vPortCleanUpTCB() will be reclaimed by ESP-IDF for internal use in v6.0.
This commit introduces the following changes...
Introduced a new CONFIG_FREERTOS_TASK_PRE_DELETION_HOOK option:
- Provides the same pre-deletion hook functionality. But users now define
vTaskPreDeletionHook() instead.
- CONFIG_FREERTOS_ENABLE_STATIC_TASK_CLEAN_UP still exists, but is marked as
deprecated. This is to maintain compatibility with existing applications
that already define vPortCleanUpTCB().
- Removed redundant --wl --wrap workaround with vPortCleanUpTCB()
- Added todo notes to remove support for user defined vPortCleanUpTCB()
completely in v6.0.
- Updated test cases to use new CONFIG_FREERTOS_TASK_PRE_DELETION_HOOK option
Freed up portCLEAN_UP_TCB() to call a new internal vPortTCBPreDeleteHook():
- vPortTCBPreDeleteHook() now replaces the previous "wrapped" implementation
of vPortCleanUpTCB().
- vPortTCBPreDeleteHook() is an internal task pre-delete hook for IDF FreeRTOS
ports to inject some pre-deletion operations.
- Internal pre-delete hook now invokes user provided vTaskPreDeletionHook()
if enabled.
- Relocated vPortTCBPreDeleteHook() to correct section in port.c
- a recent commit fixed the API deprecations,
and the BLEDevice object now needs to be accessed through a tuple.
- thus the changes were made in the code wherein the object was used,
but updating the usage in the condition when de
vname is not None got skipped.
GDB on Windows incorrectly reads EOL in the script files causing 'gdb'
action to fail.
(gdb) source .../build/gdbinit/py_extensions
(gdb) source .../build\gdbinit\symbols
add symbol table from file "...\build\bootloader\bootloader.elf"
.../build\gdbinit\symbols:6: Error in sourced command file:
Undefined command: "". Try "help".
Forcing line separator to '\n' resolved the issue
Signed-off-by: Michal Jenikovsky <jendo@jmsystems.sk>
1. The original test, before hint modules support was added, used
tempfile.NamedTemporaryFile in a way which is not supported on windows.
It was having the file open, which the hints tried to read it, leading
the EPERM exception. The docs[1] says this is not supported.
<quote>
Whether the name can be used to open the file a second time, while the
named temporary file is still open, varies across platforms (it can be
so used on Unix; it cannot on Windows)
</quote>
2. The hint module component_requirements test used the idf.py directly,
which is idf.py.exe on windows.Now it's starting idf.py through python.
We could probably used shell=True, but this approach is used in other
tests too.
Anyway the test are now passing on windows.
[1] https://docs.python.org/3/library/tempfile.html
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Following tests were added.
1. Test for missing header directory in component's INCLUDE_DIRS
2. Test for missing dependency in component's PRIV_REQUIRES
3. Test for missing dependency in component's REQUIRES
4. Test for dependency in PRIV_REQUIRES which should be in REQUIRES
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Implementation of hint module for component dependency. It can provide
hint about missing header directory in component's INCLUDE_DIRS or
about missing component dependency in REQUIRES or PRIV_REQUIRES.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently hints are supported based on hints.yml only, which may be
limiting for some use cases. This introduces a generic plugin approach,
which allows to implement hint module that doesn't require entry in hints.yml.
Such module has the full command output available and it is not limited to
a single regex in hints.yml.
Note that regex in hint.yml expects the output concatenated into a single line,
but hint modules are getting the output unchanged.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Introduces get_build_context() helper, which allows to get build context, e.g.
project description, at places where this info is not available. The
build context is set within ensure_build_directory.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
feat(idf.py): Allow adding arguments from file via @filename.txt (#11783) (GitHub PR)
Closes IDFGH-10584 and IDFGH-10539
See merge request espressif/esp-idf!24955
The ticket below reports some very strange behaviour, where
even a simple command like the following
python3 -c "import subprocess; subprocess.run([''])"
actually spawns some process and pass instead of raising
PermissionError. Even though the problem is most probably somewhere
else, not in idf_tools, we may just return early if there is no
command available for get_version().
Closes https://github.com/espressif/esp-idf/issues/11880
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
When interrupt allocation fails, esp_intr_alloc will now print a
message telling that no free interrupt was found. This message is
then checked in hints.yml, to give a link to the troubleshooting
guide.
Currently ESP_PYTHON is used in the install.sh script, but it's
overwritten in detect_python.sh. This allows to explicitly specify
python binary, which should be used to create venv.
$ ESP_PYTHON=/usr/bin/python3.7 ./install.{sh,fish}
$ ESP_PYTHON=/usr/bin/python3.7 ./export.{sh,fish}
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Now the soc interrupts definitions are scattered around in the esp-idf
which are out of sync. Put interrupts definitions in soc/periph_defs.h
(!ESP32) or soc/soc.h(ESP32) together in soc/interrupts.h.
idf.py spawns gdb process within a thread and uses Thread.join() to wait
for the gdb process to finish. As CTRL+C(SIGINT) is used by gdb to interrupt the
running program, we catch the SIGINT while waiting on the gdb to finish,
and try Thread.join() again.
With cpython's commit
commit a22be4943c119fecf5433d999227ff78fc2e5741
Author: Victor Stinner <vstinner@python.org>
Date: Mon Sep 27 14:20:31 2021 +0200
bpo-45274: Fix Thread._wait_for_tstate_lock() race condition (GH-28532)
this logic doesn't work anymore, because cpython internally marks the
thread as stopped when join() is interrupted with an exception. IMHO
this is broken in cpython and there is a bug report about this
https://github.com/python/cpython/issues/90882. Problem is that
waiting on a thread to finish is based on acquiring a lock. Meaning
join() is waiting on _tstate_lock. If this wait is interrupted, the
above referenced commit adds a logic that checks if the lock is help,
meaning the thread is done and marks the thread as stopped. But there is
no way to tell if the lock was acquired by us running join() or if it's
held by someone else e.g. still by the thread bootstrap code. Meaning
the thread is still running.
I may be missing something, but I don't see any reason why to spawn gdb
process within a thread. This change removes the thread and spawns gdb
directly. Instead waiting on a thread, we wait on the process to finish,
replacing join() with wait() and avoiding this problem.
Closes https://github.com/espressif/esp-idf/issues/11871
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Usage of `flushregs` in gdb will result in the following warning:
```
Warning: 'flushregs', an alias for the command 'maintenance flush register-cache', is deprecated.
Use 'maintenance flush register-cache'.
```
This patch updates to use the recommend command.
Signed-off-by: Daniel Mangum <georgedanielmangum@gmail.com>
Merges https://github.com/espressif/esp-idf/pull/11535
There are some register description errors in parts of rom/gpio.h
This commit update the incorrect comments in rom/gpio.h And now
esp_rom_gpio.h is recommend instead of rom/gpio.h. So this commit adds
macro SIG_GPIO_OUT_IDX in esp_rom_gpio_connect_out_signal
function and removes the reference to gpio.h in some source files.
Closes https://github.com/espressif/esp-idf/issues/11737
Verify that curses tool can be successfully imported on unix systems
When detected:
- installing esp-idf -> reinstall python environment
- using idf.py menuconfig -> raise error with hint message
Closes https://github.com/espressif/esp-idf/issues/11643
fix(hal/include): fix header violations in hal component
fix(hal/include): Move type definitions from `xx_hal.h` to `xx_types.h`
fix(hal/include): Move type definitions from `xx_hal.h` to `xx_types.h`
fix(hal/include): Add comment for a far away `#endif`
fix(hal/include): change scope for cpp guard
ci: Remove components/hal/ comment from public headers check exceptions
Add missing include macro sdkconfig.h for header files
Add missing include macro stdbool.h for header files
Add missing include macro stdint.h for header files
Add missing capability guard macro for header files
Add missing cpp guard macro for header files
Remove some useless include macros
Add some missing `inline` attribute for functions defined in header files
Remove components/hal/ from public headers check exceptions
fix(hal/include): fix invalid licenses
fix(hal/include): fix invalid licenses
fix(hal/include): add missing soc_caps.h
fix(hal): include soc_caps.h before cap macro is used
fix(hal): Remove unnecessary target check
fix(hal): fix header and macro problems
Add missing include macro
Remove loop dependency in hal
Add comment for far-away endif
fix(hal): Add missing soc_caps.h
ci: update check_copyright_ignore.txt
Change the sequence of `#include` macro, cpp guard macro
Change the wrap scope of capacity macro
fix(hal): Change position of C++ guard to pass test
- add hardware stack guard based on assist-debug module
- enable hardware stack guard by default
- disable hardware stack guard for freertos ci.release test
- refactor rtos_int_enter/rtos_int_exit to change SP register inside them
- fix panic_reason.h header for RISC-V
- update docs to include information about the new feature
Current version of the test is using "git-submodule foreach", which
requires submodules to be initialized. Non-initialized submodules are
ignored. Our CI is not performing submodule initialization, but instead
it only downloads the submodule content in tools/ci/ci_fetch_submodule.py
from cache and copies it into the submodule path.
Since we already know the submodule path from .gitconfig, we can use it
as argument to git-ls-tree and avoid calling git-submodule at all. This
allows to perform the test even if the submodules are not initialization
and also it makes the code simpler.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This uses the test from CI introduced in MR !23989. Even though the
original CI test is run with pytest, it doesn't use any pytest specific
code/features and it should never need them. So it make sense just to
re-use the code.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The script works the same way with httpd and softap mode, but it's a bit
confusing to provision the device over Ethernet or USB and call the
transport "softap". That's why we introduce an alias called httpd which
uses the same idea (http server with service name), but that service
runs on any interface (where the specified service is available)
As pointed out by Fu Hanxi, the pytest_build_system job is currently
using --parallel-index and --parallel-count, which are provided by
pytest-embedded, so we should not disable it. Moreover to properly
disable pytest-embedded we should use "no:pytest_embedded". Meaning
this probably was not working as indented anyway.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This adds SBOM information for submodules, which are not managed
by Espressif. Meaning there is no fork for them in the espressif
namespace. Other submodules should add sbom.yml manifest file to
the root of their git repository.
The SBOM information for submodules is stored in the .gitmodules file.
Each SBOM related variable has the "sbom-" prefix and the following
variables may be used:
sbom-version:
submodule version
sbom-cpe:
CPE record if available in NVD. This will be used by the SBOM
tool to check for possible submodule vulnerabilities. The
version in the CPE can be replaced with the "{}" placeholder,
which will be replaced by the "sbom-version" value from above.
sbom-supplier:
Person or organization who is providing the submodule.
It has to start with "Person:" or "Organization:" prefix
as required by the SPDX-2.2 standard.
sbom-url:
URL to the project if exists, e.g. github.
sbom-description:
Project description.
sbom-hash:
Submodule SHA as recorded in the git-tree. This field is used by
CI to check that the submodule checkout hash and info in .gitmodules
are in sync. IOW if submodule is updated and it has SBOM info in
.gitmodules, the .gitmodules has to be updated too. The test is
part of this commit. The checkout has of the submodule can be found
by using "git submodule status".
Example for micro-ecc submodule
---8<---
[submodule "components/bootloader/subproject/components/micro-ecc/micro-ecc"]
path = components/bootloader/subproject/components/micro-ecc/micro-ecc
url = ../../kmackay/micro-ecc.git
sbom-version = 1.0
sbom-cpe = cpe:2.3🅰️micro-ecc_project:micro-ecc:{}:*:*:*:*:*:*:*
sbom-supplier = Person: Ken MacKay
sbom-url = https://github.com/kmackay/micro-ecc
sbom-description = A small and fast ECDH and ECDSA implementation for 8-bit, 32-bit, and 64-bit processors
sbom-hash = d037ec89546fad14b5c4d5456c2e23a71e554966
---8<---
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This extends information provided in the project_description.json file.
Newly added information can be used in the SBOM generating tool and
also to improve hints regarding the the component dependency issues.
Added fields
version:
This adds versioning to the project_description.json file,
so it's easy to identify if it contains the required information.
project_version:
Can be used as a version for the resulting binary e.g. `hello_world.bin`.
idf_path:
This one is probably not necessary, but it allows tools to run even without
esp-idf environment exported(e.g. export.sh).
c_compiler:
The `CMAKE_C_COMPILER` value with full path to the compiler binary. This can
be used to get information about toolchain, which was used to build the project.
common_component_reqs:
List of common components as presented in cmake's __COMPONENT_REQUIRES_COMMON
and set in tools/cmake/build.cmake:__build_init().
build_component_info:
Detailed information about components used during build. It's a
dictionary with the component name as a key and each component has
a dictionary with detailed information. Following is an example for
the efuse component.
"efuse": {
"alias": "idf::efuse",
"target": "___idf_efuse",
"prefix": "idf",
"dir": "/home/fhrbata/work/esp-idf/components/efuse",
"type": "LIBRARY",
"lib": "__idf_efuse",
"reqs": [],
"priv_reqs": [ "bootloader_support", "soc", "spi_flash" ],
"managed_reqs": [],
"managed_priv_reqs": [],
"file": "/home/fhrbata/work/blink/build/esp-idf/efuse/libefuse.a",
"sources": [ "/home/fhrbata/work/esp-idf/components/efuse/esp32s3/esp_efuse_table.c", ... ],
"include_dirs": [ "include", "esp32s3/include" ]
}
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently make_json_list() returns '[ "" ]' for empty cmake list. Fix this
so empty json list is returned instead.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
- This features allows the NVS encryption keys to be derived and protected using
the HMAC peripheral. Since the encryption keys are derived at runtime, they
are not stored anywhere in the flash and hence this feature does not require
a separate `nvs_keys` partition.
This commit adds support for handling multiple requests simultaneously by introducing two new functions: `httpd_req_async_handler_begin()` and `httpd_req_async_handler_complete()`. These functions allow creating an asynchronous copy of a request that can be used on a separate thread and marking the asynchronous request as completed, respectively.
Additionally, a new flag `for_async_req` has been added to the `httpd_sess_t` struct to indicate if a socket is being used for an asynchronous request and should not be purged from the LRU cache.
An example have been added to demonstrate the usage of these new functions.
Closes https://github.com/espressif/esp-idf/issues/10594
Signed-off-by: Harshit Malpani <harshit.malpani@espressif.com>
esp_bootloader_format: Adds bootloader description structure to read bootloader version from app
Closes IDFGH-7206 and IDFGH-7573
See merge request espressif/esp-idf!21592
spi_flash: One more step for supporting flash suspend. 1. Support more esp chips 2. Improve real-time performance 3. Make timing more stable
See merge request espressif/esp-idf!22755
Currently hints are processed only once the process is finished and
exits with non-error exit code. In interactive mode, e.g. for monitor,
we want to process ouput lines for hints right away.
This adds a simple buffer, which keeps the last line and once EOL is
reached, it is processed for hints.
Since the original hints processing was file based, a new helper
function was added to allow processing hints in string.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
* reverted old faulty mechanism to set different
standard according to the toolchain
* Using -std=gnu++2b now for both gcc and clang
* Added a build test app to check the C++ standard in IDF
* Updated english docs to reflect the change to C++23
This commit refactors the "freertos" component's structure as follows:
- "FreeRTOSConfig.h" related files moved to "./config" directory
- Refactored CMakeLists.txt file in preparation for v10.5.1 upgrade
- Grouped list appends based on component organization
- Removed some unecessarily public "include_dirs"
- Removed FreeRTOS-openocd.c
- uxTopUsedPriority has been added back to tasks.c since v10.4.2
- Thus the workaround in FreeRTOS-openocd.c is no longer needed and can
be removed.
ADDITIONAL_MAKE_CLEAN_FILES is deprecated and only worked with make.
Replaced with the new ADDITIONAL_CLEAN_FILES (CMake 3.15) which also works with ninja.
Changed rv_utils_intr_edge_ack and esp_cpu_intr_edge_ack to
take uint32_t instead of int to avoid build errors.
The test is to test in particular that __builtin_ffsll, used in
xt_utils.h, which is included via esp_cpu.h, compiles fine
in C++20 with -Wsign-conversion enabled.
Closes https://github.com/espressif/esp-idf/pull/10895
* The unit tests are derived from the FreeRTOS test app
in components/freertos/test_apps/freertos. They are
quite incompatible with the main test application, which
is why they have been placed under
tools/test_apps/linux_compatible/linux_freertos for now.
This feature allows calling fsync even if the vfs component is not
used.
The second part of the commit adds an fsync call in the panic test app
enabling it to be used over usb-serial-jtag.
system: support USB_OTG CDC console on ESP32-S3
Closes IDF-2048, IDF-2987, IDFGH-7134, IDFGH-7291, and IDFGH-7835
See merge request espressif/esp-idf!19312
Used esp-rom tag esp32s3-20210327 and did manual cleanup.
Rename s_usb_osglue to rom_usb_osglue like it was done for esp32s2.
Some comments in esp32s2 headers are synced from esp32s3.
On Windows, when path is specified as absolute for cmdl argument, cmake can
interpret parts of the path as invalid escape chars. For example "C:\Users\..."
will result in "Invalid character escape '\U'." Externally specified
paths should be converted into cmake's representation, which uses '/'.
This can be done e.g. by using 'get_filename_component()'. Currently
there doesn't seem to be any problem with this, but let's add a test for
this.
Suggested-by: Ivan Grokhotkov <ivan@espressif.com>
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently the forced progress in RunTool is trying to fit the output
line into a terminal width, but it doesn't take into an account a
situation when the terminal width is reported as zero. This manifests
when running in docker image with redirected output and can be seen
in the github workflow output for esp-idf-ci-action.
docker run --rm -t my_ubuntu_esp python3 -c 'import os,sys; print(os.get_terminal_size(), sys.stdout.isatty())'
os.terminal_size(columns=238, lines=59) True
vs
docker run --rm -t my_ubuntu_esp python3 -c 'import os,sys; print(os.get_terminal_size(), sys.stdout.isatty())' | tee
os.terminal_size(columns=0, lines=0) True
Since the output is reported as tty and the terminal width as 0, the
fit_text_in_terminal() function returns empty string. I also verified this
by running idf.py build inside a testing docker image.
This fix adjusts the fit_text_in_terminal() function to return original
line if the terminal width is zero.
Also simplify the progress print and use same approach as ninja
https://github.com/ninja-build/ninja/blob/master/src/line_printer.cc#L66
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently if the IDF_TARGET env is set, and old sdkconfig exists with
different target value in CONFIG_IDF_TARGET, the set-target action fails
complaining about the IDF_TARGET env and value in sdkconfig being different.
We should ignore IDF_TARGET value from sdkconfig, because we are
actually setting new target and the old sdkconfig is renamed in cmake.
This can be easily reproduced with
---8<---
$ IDF_TARGET=esp32 idf.py set-target esp32
$ IDF_TARGET=esp32s3 idf.py set-target esp32s3
Project sdkconfig '/home/fhrbata/work/hello_world/sdkconfig' was generated
for target 'esp32s3', but environment variable IDF_TARGET is set to 'esp32'.
Run 'idf.py set-target esp32' to generate new sdkconfig file for target esp32.
---8<---
This also adds test for this use case to test_non_default_target.py.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit removes all idf_size.py files and references to them and adds esp-idf-size as a dependency and adequate wrappers to avoid breaking changes.
Two new tests are added.
1) test_check_python_dependencies
The test prepares artificial constraints file containing packages from
requirements.core.txt, which are also reported in pip-freeze output
for virtual env. The constraints file requires package versions higher
than currently installed in venv, so check_python_dependencies
should fail for all of them.
2) test_check_required_packages_only
Test for espressif/esp-idf/-/merge_requests/17917. After installing
core requirements, install additional foopackage, which is embedded.
Add version requirement for foopackage, which cannot be satisfied,
to constraints file. Since foopackage is not a direct requirement,
check-python-dependencies should not fail.
This also fixes existing TestCustomPythonPathInstall test, which sets
IDF_PYTHON_ENV_PATH, but does not restore it. Unittest seems to be
running tests in order based on class/test name. Meaning this test runs
before TestPythonInstall and all tests in TestPythonInstall are using
the latest tmpdir from TestCustomPythonPathInstall as IDF_PYTHON_ENV_PATH.
IOW TestPythonInstall is actually testing custom python env path, same as
TestCustomPythonPathInstall. This fixes it by restoring the
IDF_PYTHON_ENV_PATH. Note that since the actual IDF_PYTHON_ENV_PATH
was different(tmpdir) from PYTHON_DIR, the default PYTHON_DIR was never used,
so the tests were running with old python env(no
shutil.rmtree(PYTHON_DIR).
Since TestCustomPythonPathInstall is inheriting from TestPythonInstall
it also runs test_opt_argument and test_no_constraints for the second
time. This seems unnecessary, so this patch also skips these two tests
in TestCustomPythonPathInstall.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
esp_execute_shared_stack_function always restored the stack watchpoint
regardless of CONFIG_FREERTOS_WATCHPOINT_END_OF_STACK. This would lead
to an abondoned but active watchpoint on a former stack once the task
calling esp_execute_shared_stack_function is deleted, if
CONFIG_FREERTOS_WATCHPOINT_END_OF_STACK is inactive.
This has been fixed now.
Closes https://github.com/espressif/esp-idf/issues/10414
