Warn if IDF_PYTHON_ENV_PATH set to a suspicious path with
different ESP-IDF version and/or Python version.
Fail if the virtual environment was created for a different ESP-IDF
version.
Closes https://github.com/espressif/esp-idf/issues/13196
This commit introduce SOC_MEM_NON_CONTIGUOUS_SRAM flag (that enebled for
esp32p4). If SOC_MEM_NON_CONTIGUOUS_SRAM is enabled:
- LDFLAGS+=--enable-non-contiguous-regions
- ldgen.py replaces "arrays[*]" from sections.ld.in with objects under
SURROUND keyword. (e.g. from linker.lf: data -> dram0_data SURROUND(foo))
- "mapping[*]" - refers to all other data
If SOC_MEM_NON_CONTIGUOUS_SRAM, sections.ld.in file should contain at
least one block of code like this (otherwise it does not make sense):
.dram0.bss (NOLOAD) :
{
arrays[dram0_bss]
mapping[dram0_bss]
} > sram_low
.dram1.bss (NOLOAD) :
{
/* do not place here arrays[dram0_bss] because it may be splited
* between segments */
mapping[dram0_bss]
} > sram_high
Bugfix/nvs Improved handling of BLOB during unreliable power environment and concurrent data access scenarios
Closes IDF-8839
See merge request espressif/esp-idf!28843
This issue will happen to multi-dut test cases
- without `target` defined in param
- and with `app_path` defined in param
- and with `pytest.mark.target` markers
This commit replaces the use of portNUM_PROCESSORS and configNUM_CORES
macros in all of ESP-IDF. These macros are needed to realize an SMP
scenario by fetching the number of active cores FreeRTOS is running on.
Instead, a new Kconfig option, CONFIG_FREERTOS_NUMBER_OF_CORES, has been
added as a proxy for the FreeRTOS config option, configNUMBER_OF_CORES.
This new commit is now used to realize an SMP scenario in various places
in ESP-IDF.
[Sudeep Mohanty: Added new Kconfig option CONFIG_FREERTOS_NUMBER_OF_CORES]
Signed-off-by: Sudeep Mohanty <sudeep.mohanty@espressif.com>
Some additional checks related to secure version of the application in
anti-rollback case have been added to avoid any attempts to boot lower
security version but valid application (e.g., passive partition image).
- Read secure_version under sha256 protection
- First check has been added in the bootloader to ensure correct secure
version after application verification and loading stage. This check
happens before setting up the flash cache mapping and handling over
the final control to application. This check ensures that application
was not swapped (e.g., to lower security version but valid image) just
before the load stage in bootloader.
- Second check has been added in the application startup code to ensure
that currently booting app has higher security version than the one
programmed in the eFuse for anti-rollback scenario. This will ensure
that only the legit application boots-up on the device for
anti-rollback case.
- Updated WiFi libraries based on the IDF changes
- Renamed esp_wifi_native.h to esp_wifi_types_native.h
- Reverted forward declarations of native wifi types and HE types
- Forward declare as opaque structs only if we don't have native types (csi and ctrl_packet types)
Currently we silently ignore when the original component is not found
in a hope we can provide at least some meaningful hint. As it turned
out it's not true. Instead of providing misleading hint, just return
error. This adds several checks for situations, which should not happen,
but when they do it should be easier to identify the root cause of the
problem.
For example when hint module received malformed output with extra new
lines, e.g. caused by a bug in RunTool, it wrongly reported the original
component as source component.
This should also fix the tests on Windows.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, only `.o`, `.*.obj` and `.obj` patterns
are taken into account. It would be great to have
object files with the `.*.o` extension pattern
(e.g. `file.cpp.o`) also processed as they're quite
widespread in third-party integrations.
Currently RunTool reads command's output with asyncio read, which
returns bytes. This is decoded into python's string and the output already
contains OS specific line endings, which on Windows is CRLF. Problem is
that the command output is saved by using python's text stream/file,
which replaces LF, native python's line ending, with OS specific line ending.
On Windows, and in this particular case, the CRLF from the command output is
translated into CRCRLF and saved in the commands output file. When this
file is read in again, e.g. for hint modules, the CRCRLF is replaced
with LFLF. Again the file is open as text file. Meaning a new emply line
is added.
Fix this by opening the output file with "newline=''", which prevents
this translation. We already have the OS specific line ending in the
command's output.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Some files that should have their copyrights checked are still placed on the
copyright ignore list.
- These entries have been tidied up
- Copyrights of those files have been updated.
This header has been removed for the following reasons:
- Header is misplaced. 'xxx_types.h' headers should be placed in the 'hal'
component.
- The 'usb_xxx_endpoint_t' should be placed in the 'xxx_struct.h' header.
usb_pins.h and usb_periph.h/c lists mappings of USB DWC signals to GPIOs used
to connect to external FSLS PHYs. However, those signals can be routed to any
GPIOs via the GPIO matrix. Thus, these mapping are meaningless and have been
deprecated.
While collecting, we stop registering the new plugin.
Otherwise the new created plugin will override the one we passed
also run the tests inside idf. the behavior is different.
build:
- upgrade idf-build-apps to 2.x
- unify get_pytest_apps and get_cmake_apps to get_all_apps
- returns (test_apps, non_test_apps) tuple
- add tests for the new get_all_apps
assign:
- generate build report
- generate target test pipeline based on the build report
target test:
- download artifacts from minio server
- users can use `pytest --pipeline-id xxxxx` to download and flash
the binaries from the artifacts
.post:
- generate target test reports
CONFIG_BOOTLOADER_COMPILER_OPTIMIZATION_NONE doesnt really have many use cases, but it will
force us to keep increasing bootloader segment sizes just to allow for building this options.
Deprecate this config for new chips.
fix(panic): fixed cache error being reported as illegal instruction
Closes IDF-6398, IDF-5657, IDF-7015, and IDF-6733
See merge request espressif/esp-idf!27430
For Linux builds if the embedded data length exceeded 16-bit value then
the build used to fail with following error:
build/x509_crt_bundle.S: Assembler messages:
build/x509_crt_bundle.S:4201: Warning: value 0x1056e truncated to 0x56e
GNU ASM for X86 systems treats .word attribute as of size 2 bytes, this commit
uses .long attribute to take the size to 4 bytes.
1. Rename ESP_SLEEP_DEEP_SLEEP_WAKEUP_DELAY to ESP_SLEEP_WAIT_FLASH_READY_EXTRA_DELAY
2. Set ESP_SLEEP_WAIT_FLASH_READY_EXTRA_DELAY visible for all targets
fix(tools): replace nvs_partition_gen.py with new esp-idf-nvs-partition-gen package
Closes IDF-8147 and IDF-1958
See merge request espressif/esp-idf!26758
feat(bootloader_support): Make `esp_secure_boot_verify_sbv2_signature_block()` function public
Closes IDFGH-11599
See merge request espressif/esp-idf!27742
The size of partition of type APP should be multiple of 4 KB. Partition
generation tool now make this as a mandatory requirement. This is
minimum flash erase size. If the size of the APP type partition is not
aligned to 4 KB then the last erase operation could go beyond the allocated
partition and hence may fail. This issue would only be observed when the
firmware size grows very close to the allocated partition size, and hence
causing the OTA update to fail.
For already deployed devices on-field with the size of APP partition not
aligned to flash sector boundary, it is best to ensure that firmware
size always remains within the lower 4 KB boundary of the total
allocated space. While migrating to ESP-IDF 5.3 release, partition table
for an existing project can be adjusted accordingly for the build to
succeed.
Found during discussion in https://github.com/espressif/esp-idf/pull/12460
When idf.py coredump-debug is launched with '--core' argument, it
tries to determine the file format (raw, elf, b64). To detect the
'raw' core dump the code checked if the version word matched one of
the known values.
However, the version word also contains the chip ID in the high
half-word, so the check failed for anything other than the ESP32.
The detection of core file format has been moved to esp-coredump
package in version 1.9.0, including the fix for chip ID.
Reported in https://github.com/espressif/esp-idf/issues/10852
Currently sbom manifest is checked only in .gitmodules and
this check is done in pre-commit and also in CI. Meaning it's running
three times(pre-commit before push if user has it enabled, in CI
as there is the pre-commit run again and again with test in CI). Since
esp-idf-sbom contains a full manifest validation support and pre-commit
plugin for it, let's use it. This removes all the current sbom testing
and replaces it with a signle pre-commit plugin which validates all
manifests files(sbom.yml, idf_component.yml, .gitmodules and also
referenced manifests) in repository. Note that this checks all
manifests, not only ones which were modified. The check is reasonably
fast though, so it should not cause any problem. The reason for
validating all manifest files is that we want to make sure that the sbom
information in .gitmodules is updated too and that the hash
recorded in .gitmodules is up-to-date. Meaning submodule update
would not trigger this plugin, because no manifest was changed.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit enables the standad VERSION argument for the project() macro
in ESP-IDF. The VERSION argument is compilant with the requirements of
cmake 3.16. This commit also adds new test cases for verifying the
validity of the version argument.
Merges https://github.com/espressif/esp-idf/pull/12461
Co-authored-by: Sudeep Mohanty <sudeep.mohanty@espressif.com>
On riscv chips accessing cache mapped memory regions over the ibus would
result in an illegal instructions exception triggering faster than the cache
error interrupt/exception.
Added a cache error check in the panic handler, if any cache errors are active
the panic handler will now report a cache error, even if the trigger exception
was a illegal instructions.
This extension allows running programs in QEMU similar to running
them on a real chip:
- 'idf.py qemu' — builds and runs the program in QEMU. User gets
a QEMU instance launched, and can work with it as a normal QEMU
instance.
- 'idf.py qemu monitor' — same, but QEMU is launched in the
background, and idf_monitor runs in the foreground, showing QEMU
output. Compared to only running 'idf.py qemu' this enables, for
example, automatic backtrace decoding.
- 'idf.py qemu gdb' — launches QEMU in the background and opens an
interactive GDB prompt, connecting it to QEMU.
- 'idf.py qemu --gdb monitor' and 'idf.py gdb' in another shell:
launches QEMU in the background, keeps it suspended until GDB is
connected, and opens idf_monitor. GDB can be used in another shell
to debug the application.
IDF FreeRTOS v10.5.1 no longer accepts out of range xCoreID arguments in
"PinnedToCore" task creation functions when building for single-core. This
commit fixes those violations through ESP-IDF.
change(usb_serial_jtag): Add example for usb serial jtag echo
Closes IDFGH-11503, IDFGH-11493, and IDFGH-11478
See merge request espressif/esp-idf!27366
This commit deprecates the "freertos/xtensa_context.h" and "xtensa/xtensa_context.h"
include paths. Users should use "xtensa_context.h" instead.
- Replace legacy include paths
- Removed some unnecessary includes of "xtensa_api.h"
- Add warning to compatibility header
With 8959555cee7e[1] ("setup_git_directory(): add an owner check for the top..")
git added an ownership check of the git directory and refuses to
run any git commands, even parsing the config file, if the git directory
is not owned by the current user. The "fatal: detected dubious ownership in repository"
is reported.
This fixes CVE-2022-24765[2], which allows to compromise user account. On a
multi-user system or e.g. on a shared file system, one user may create a "rogue"
git repository with e.g. core.fsmonitor set to an arbitrary command. Other user
may unwillingly execute this command by running e.g. git-diff or
git-status within the "rogue" git repository, which may be in one of the parent
directories. If e.g. PS1 is set to display information about a git
repository in CWD, as suggested in Git in Bash[3], the user do not need to run
any git command to trigger this, just entering some subdirectory under
this "rogue" git repository is enough, because the git command will be
started transparently through the script used in PS1. The core.fsmonitor
can be set to arbitrary command. It's purpose is to help git to identify changed files
and speed up the scanning for changed files.
rogue
├── .git # owned by user1
└── dir1 # owned by user2
├── dir2 # owned by user2
└── .git # owned by user2
user1 sets core.fsmonitor for git repository in rogue directory
$ git config --add core.fsmonitor "bash -c 'rm -rf \$HOME'"
user2 enters dir1 and runs e.g. git diff and triggers the core.fsmonitor command.
The ownership check may cause problems when running git commands in
ESP-IDF Docker container. For example user may run the container as
root, but the mounted project may be owned by a particular user.
In this case git will refuse to execute any git command within the
"/project" directory, because it's not owned by root. To overcome this,
git allows to set safe.directories, for which the ownership check is
skipped. The security check may be completely disabled by setting
safe.directories to "*". This solution was proposed in PR 12636[4], but
it would allow make it possible to exploit this vulnerability again.
This fix allows user to specify git's safe.directory in IDF_GIT_SAFE_DIR
environmental variable, which may be set during container startup.
The IDF_GIT_SAFE_DIR has same format as PATH and multiple directories can be
specified by using a ":" separator. To entirely disable this git security check
within the container, user may set IDF_GIT_SAFE_DIR='*'. This might be
heplfull in CI.
Closes https://github.com/espressif/esp-idf/pull/12636
[1] - 8959555cee
[2] - https://nvd.nist.gov/vuln/detail/cve-2022-24765
[3] - https://git-scm.com/book/en/v2/Appendix-A%3A-Git-in-Other-Environments-Git-in-Bash
[4] - https://github.com/espressif/esp-idf/pull/12636
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This integrates esp_idf_size.ng, refactored esp-idf-size version, into
esp-idf and enables it by default. The esp_idf_size.ng may be enabled
by using the --ng option, but also via ESP_IDF_SIZE_NG env. variable,
which is used in this integration.
New -l/--legacy option is added, which enforces usage of the old version.
This option can be also set via "ESP_IDF_SIZE_LEGACY" env. variable.
This should allow to easily switch back to old version if there is any
problem.
The new version is used by default for all formats, except for the "json".
Examples:
$ idf.py size # uses refactored version
$ idf.py size --legacy # uses legacy version
$ idf.py size --l # uses legacy version
$ idf.py size --format json # uses legacy version
$ idf.py size --format json2 # uses refactored version
$ export ESP_IDF_SIZE_LEGACY="1" # use legacy version only from now on
ESP_IDF_SIZE_FORCE_TERMINAL, which forces terminal control codes(colors), is also set
when running from idf.py, so the colors are propagated even if stdout
for esp_idf_size.ng is not attached to tty.
The same changes are applied also to the idf_size.py wrapper.
There is an import check if esp_idf_size.ng is available. If not,
we switch into the legacy mode. This should cover situation when the
esp-idf has support for refactored version, but it's not installed.
This should also allow users to bind to a legacy version(<1.0.0) and the
idf.py size and idf_size.py should still work. This also allow us to
restring the version in constraints file if we need to switch back to
legacy version globally.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
