Two new tests are added.
1) test_check_python_dependencies
The test prepares artificial constraints file containing packages from
requirements.core.txt, which are also reported in pip-freeze output
for virtual env. The constraints file requires package versions higher
than currently installed in venv, so check_python_dependencies
should fail for all of them.
2) test_check_required_packages_only
Test for espressif/esp-idf/-/merge_requests/17917. After installing
core requirements, install additional foopackage, which is embedded.
Add version requirement for foopackage, which cannot be satisfied,
to constraints file. Since foopackage is not a direct requirement,
check-python-dependencies should not fail.
This also fixes existing TestCustomPythonPathInstall test, which sets
IDF_PYTHON_ENV_PATH, but does not restore it. Unittest seems to be
running tests in order based on class/test name. Meaning this test runs
before TestPythonInstall and all tests in TestPythonInstall are using
the latest tmpdir from TestCustomPythonPathInstall as IDF_PYTHON_ENV_PATH.
IOW TestPythonInstall is actually testing custom python env path, same as
TestCustomPythonPathInstall. This fixes it by restoring the
IDF_PYTHON_ENV_PATH. Note that since the actual IDF_PYTHON_ENV_PATH
was different(tmpdir) from PYTHON_DIR, the default PYTHON_DIR was never used,
so the tests were running with old python env(no
shutil.rmtree(PYTHON_DIR).
Since TestCustomPythonPathInstall is inheriting from TestPythonInstall
it also runs test_opt_argument and test_no_constraints for the second
time. This seems unnecessary, so this patch also skips these two tests
in TestCustomPythonPathInstall.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
esp_execute_shared_stack_function always restored the stack watchpoint
regardless of CONFIG_FREERTOS_WATCHPOINT_END_OF_STACK. This would lead
to an abondoned but active watchpoint on a former stack once the task
calling esp_execute_shared_stack_function is deleted, if
CONFIG_FREERTOS_WATCHPOINT_END_OF_STACK is inactive.
This has been fixed now.
Closes https://github.com/espressif/esp-idf/issues/10414
Update wifi lib with below -
1. Create NAN Discovery SM for beaconing & cluster formation
2. Create NAN interface for Tx/Rx of beacons & action frames
3. Add commands & events for NAN Services Publish/Subscribe/Followup
4. Add NAN Datapath definitions, Events, Peer structures
5. Support for forming and parsing of Datapath related attributes
6. Modules for NDP Req, Resp, Confirm, Term, Peer management
7. NAN Interface related additions in Datapath, Data Tx Q's
In addition include below changes -
1. Add netif and driver support for NAN Interface
2. Add simple examples for Publisher-Subscriber usecases
3. Add an advanced console example that supports commands
for NAN Discovery, Services & Datapath
4. Add wifi_apps for providing better NAN API's and Peer management
Co-authored-by: Shyamal Khachane <shyamal.khachane@espressif.com>
Currently the set-target has sdkconfig file name hardcoded to the
default one and doesn't honor custom config paths or names.
IMHO the only place where we can really now the config file name
is in cmake. But also the config should be really renamed only if
the set-target action is running.
This moves the config file renaming into cmake and it's performed only
when _IDF_PY_SET_TARGET_ACTION env. var. is set to 'ON'. This should
hopefully guarantee that it's really renamed only while set-target is
running.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Extend target checks in cmake, in case it's run directly and not via
idf.py or if idf.py misses something. This may happen
for example if cmake variables are set in project's CMakeLists.txt.
Some clean-ups are included along with the new checks and tests.
1. __target_check() function is removed. IIUC it should never fail,
because the selected target is explicitly passed as environmental
variable to kconfgen. Meaning the IDF_TARGET from environment variable may
not be actually used in kconfgen if IDF_TARGET is already set it cmake cache.
Note that the IDF_TARGET environment variable used for kconfgen is not
based on the actual IDF_TARGET environment variable set for idf.py, but
rather on the value set in __target_init() with
set(IDF_TARGET ${env_idf_target} CACHE STRING "IDF Build Target")
My understanding is that the original check was introduced to handle
situation, where IDF_TARGET was already set in cmake's cache and
the IDF_TARGET from environment variable was different. Since
the kconfgen would use the original environment variable(not
explicitly passed as it is now) the IDF_TARGET in cmake and in
sdkconfig could differ. IOW I think the original check was introduced
to cope with the following cmake behaviour
set(VARIABLE "value1" CACHE STRING "test variable")
set(VARIABLE "value2" CACHE STRING "test variable")
message("Variable value: ${VARIABLE}")
output: Variable value: value1
2. I scratched by head how it is possible that the following code
in __target_check()
if(NOT ${IDF_TARGET} STREQUAL ${env_idf_target})
could fail if IDF_TARGET is not set. For example in clean project
IDF_TARGET=esp32 idf.py reconfigure
Here env_idf_target==esp32 and IDF_TARGET is not set, so I would
expect that cmake will fail with error message that the cache
and env target do not match. The thing is that the variable
evaluation is done before the if command, so it actually
sees this
if(NOT STREQUAL esp32)
which is false and the error is not printed. It can be seen
with 'cmake --trace-expand' command. I don't know if this
was used on purpose or it worked just as a coincidence, but
I find it very confusing, so I added explicit check if the
IDF_TARGET is defined before the actual check. Same for
CMAKE_TOOLCHAIN_FILE.
3. Error messages are not formated(line-wrapped) by cmake's markup
so it's easier to check the output in tests.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Extend existing target consistency checks for the two following cases.
1. Target does not match currently used toolchain
$ IDF_TARGET=esp32s2 idf.py reconfigure
$ idf.py -DIDF_TARGET=esp32c3 build
2. Target is ambiguous, because it's specified also as env. var.
IDF_TARGET=esp32s3 idf.py set-target esp32c2
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The _guess_or_check_idf_target() function has sdkconfig and sdkconfig.defaults
file names hardcoded. Since config file names may be specified with SDKCONFIG
or SDKCONFIG_DEFAULTS cmake vars, directly in CMakeLists.txt or passed in with
the -D cmake option, they are not respected.
Problem is when SDKCONFIG or SDKCONFIG_DEFAULTS is set in
CMakeLists.txt. While idf can detect cmake vars passed through it
to cmake via the -D option, detecting SDKCONFIG and SDKCONFIG_DEFAULTS
vars settings in CMakeLists.txt would require to parse it. This seems
like error prone approach. Also if the vars defined by the -D option
are passed directly to cmake, not via idf, they will not be visible to idf.
It seems reasonable to move the logic into cmake, where we know the correct
SDKCONFIG and SDKCONFIG_DEFAULTS values. So the IDF_TARGET detection/guessing
is moved into targets.cmake, where the IDF_TARGET is actually set. The target
is guessed based on the following precendence.
1) $ENV{IDF_TARGET}
2) IDF_TARGET
3) SDKCONFIG
4) sdkconfig
5) SDKCONFIG_DEFAULTS if non-empty or
$ENV{SDKCONFIG_DEFAULTS} if non-empty or
sdkconfig.defaults
6) esp32
All config files referred in $ENV{SDKCONFIG_DEFAULTS} and SDKCONFIG_DEFAULTS
are searched, compared to the current behaviour. First target found in the
above chain is used.
The original _guess_or_check_idf_target() is renamed to _check_idf_target() and
used for the target consistency checks only.
The get_sdkconfig_filename() helper is now used to get the sdkconfig file
for consistency checks. It looks in SDKCONFIG specified with the -D
option and project_description.json.
With this change config full paths are reported in messages, so it's clear
e.g. from which config the target was guessed from or which config has
consistency problem. test_non_default_target.py was adjusted to this
change and also new test for testing the IDF_TARGET guessing was added.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Get project's current sdkconfig file name. It looks in SDKCONFIG cmake var
defined by the -D option and project_description.json. If not found return
default sdkconfig.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This parses cmakes cache vars defined on command line with -D options
into dictionary. It allows to simplify the check for new cache entries
and also can be re-used for other checks.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
1. Remove RTC_CLOCK_BBPLL_POWER_ON_WITH_USB Kconfig option
During sleep, BBPLL clock always gets disabled
esp_restart does not disable BBPLL clock, so that first stage bootloader log can be printed
2. Add a new Kconfig option PM_NO_AUTO_LS_ON_USJ_CONNECTED
When this option is selected, IDF will constantly monitor USB CDC port connection status.
As long as it gets connected to a HOST, automatic light-sleep will not happen.
Closes https://github.com/espressif/esp-idf/issues/8507
Pros:
- Using thread would face GIL issue and turns out very slow when running
with poor hardware.
Cons:
- Does not support windows anymore. For testing purpose, it's fine.
- Add support for esp32s2, esp32c3 and esp32c2 for the `memprot`-related tests
- Preliminary support for esp32s3 has also been added,
the test app will be enabled for esp32s3 later when
the memprot-related issues are fixed.
- Override panic handler to dump the violation intr status
- Dump the `memprot` violation registers before calling the
real panic handler
- Handle `Illegal Instruction` exception in case of memprot permission violation
* In esp32c3 with `memprot` enabled, if we try to execute arbitrary code
from RTC_FAST_MEM we get an `Illegal Instruction` exception from the panic
handler rather than a `Memory Protection Fault`.
* This is because the Illegal Instruction interrupt occurs earlier than the
memory protection interrupt due to a higher interrupt latency.
- Added minor improvements to `panic` test app
* Replaced existing API to disable flash cache which did not disabled cache always
(`esp_flash_default_chip->os_func->start(esp_flash_default_chip->os_func_data)`)
with `spi_flash_enable_interrupts_caches_and_other_cpu`
* Included some required headers explicitly (`esp_memory_utils.h` and `esp_heap_caps.h`)
IDF_PYTHON_ENV_PATH is the path where the Python environment is created
and used. By default it is inside IDF_TOOLS_PATH. IDF_PYTHON_ENV_PATH
was exported by idf_tools.py but was not imported back. This fixes the
issue and ESP-IDF will honor the value of IDF_PYTHON_ENV_PATH.
Closes https://github.com/espressif/esp-idf/issues/10489
This adds a new outdated option, which only lists outdated
packages installed in IDF_TOOLS_PATH. It searches for the
latest installed tool version in the IDF_TOOLS_PATH/tools path and
compares it against the latest available version in the tools.json
file. If the latest version of a tool installed in IDF_TOOLS_PATH/tools
is smaller, it's reported as outdated. Nothing is reported if the tool
is up to date.
Two new tests are added. First just checks if nothing is reported in
case there is no update available. The second artificially generates
new tools.json file called tools.outdated.json and sets XTENSA_ESP32_ELF
version to 'zzzzzz'. It then checks if the XTENSA_ESP32_ELF tool
is reported as outdated by the 'zzzzzz' version.
Description of the new outdated option is addedd to docs as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This moves one hint, which was hardcoded in debug_ext.py to
hints.yml and adds a new one when openocd process does not have
permissions to the USB JTAG/serial device.
Also hint replacing the original 'Please check JTAG connection!'
hardcoded message is added.
Suggested-by: Alexey Lapshin <alexey.lapshin@espressif.com>
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The debug targets are currently not utilizing hints, because they
are not using RunTool() helper from tools.py to spawn sub-processes.
Adjusting debug targets to use RunTool() would require some significant
changes to debug targets and RunTool() as well. Since debug targets
are already storing their output in logs, we can use these and process
them for hints.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
cpu retention: add riscv core sleep critical and non-critical register layout structure definition
cpu retention: add assembly subroutine for cpu critical register backup and restore
cpu retention: add cpu core critical register context backup and restore support
cpu retention: add cpu core non-critical register context backup and restore support
cpu retention: add interrupt priority register context backup and restore support
cpu retention: add cache config register context backup and restore support
cpu retention: add plic interrupt register context backup and restore support
cpu retention: add clint interrupt register context backup and restore support
cpu retention: wait icache state idle before pmu enter sleep
Examples for zigbee are using custom license, which is not present
on the SPDX license list. This was discussed on !16205 and a decision
was made that the examples will be put on check-copyright's ignore list.
SPDX has LicenseRef-[idstring] identifier for such cases, so let's try
to use it. In this particular case the LicenseRef-Included is used to
express that the full license text is included in the source file.
Note that the LicenseRef-Included is not part of SPDX. It's just
something I used. No change is needed on the check-copyright side.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently loading of esp32s3 ROM ELF symbols fails with
"Cannot access memory at address 0x3ff194ad". Let's perform
add-symbol-file after connection to the target.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit fixes an issue where paths on Windows are case insensitive, for instance when setting the build folder its name would be converted to lowercase.
The culprit is our realpath() function, that was calling os.path.normcase() internally, since we are removing that call it makes sense to just remove the function entirely and call os.path.realpath() wherever necessary.
Closes https://github.com/espressif/esp-idf/issues/10282
The main idf.py process has a handler for SIGINT, which actually just
ignores it. The get_default_serial_port() function is called within the
idf.py context to detect port for several tools(monitor,flash) and it's
not possible to terminate it because of this. Let's ignore SIGINT only
while running idf_monitor, which uses it to spawn gdb.
Fixes: c6e3eb0922 ("idf.py.exe changes to handle Ctrl+C in correct way. H..")
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit fixes two issues:
1. Part of the header containing annotations for curr, reference and diff values for archive diffs wasn't formatted correctly
2. Only one third of the values in the table were displayed because of the wrong line format
Since the introduction of PCAddressMatcher, the executable produced
by the build system is passed to elftools.elf.elffile.ELFFile.
However on macOS, native executables are not ELF files, so the
ELFFile class raises a rather unhelpful AssertionError exception.
Given that the rest of the idf_monitor.py doesn't have assumptions
that the "elf_file" argument is an ELF file (rather than just an
executable), check if the file is a real ELF file inside
PCAddressMatcher.
When multiple targets are specified, e.g. idf.py flash monitor, the
automatic port detection is performed twice. Keep the port value in
args.port and avoid multiple calls to get_default_serial_port().
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit migrates the esp_pm unit tests from the legacy unit test to a
stand alone test app. The following CI configurations are provided
- Default: Automatic light sleep with mostly default configurations
- Options: Enables all of the optional esp_pm features
- Limits: Limit tests esp_pm
Set CONFIG_NEWLIB_TIME_SYSCALL_USE_HRT to enable to force CONFIG_ESP_TIME_FUNCS_USE_RTC_TIMER to disable
and hence prevent the following tests from being executed:
- Timestamp after abort is correct in case RTC & High-res timer have + big error
- Timestamp after restart is correct in case RTC & High-res timer have + big error
- Timestamp after restart is correct in case RTC & High-res timer have - big error
This commit removes the dependency on portUSING_MPU_WRAPPERS on the Xtensa port
of IDF FreeRTOS. This dependency was added due to a hack implemented in the
upstream port that required the usage of the "xMPUSettings" member of the TCB.
The "xMPUSettings" would be used as a pointer to the task's coprocessor save
area on the stack, even though FreeRTOS MPU support was not available.
The hack has now been removed, and the CPSA pointer is now calculated using
a combination of constant offsets values and the pxEndOfStack member of the
TCB.
Note: This impelemtation was copied from the Xtensa port of Amazon SMP FreeRTOS.
Previously get_idf_build_env didn't include the default environment
(os.environ) in its output, so the environment only contained the
variables returned by "idf_tools.py export".
If PATH already contains all the right paths, "idf_tools.py export"
doesn't return the PATH variable.
Therefore the environment returned by get_idf_build_env was usually
incomplete. Fix by combining the output of "idf_tools.py export" with
os.environ.
esp_netif various fixes: dependency cleanup, default flags, CI rules
Closes IDF-5550, IDFGH-7023, IDF-1261, and IDF-2155
See merge request espressif/esp-idf!20301
It is required to define a mandatory dependency on lwip, so we
introduced esp_netif_stack component and made it require lwip, instead
of directly depending on lwip.
This enables building w-out lwip and support other TCP/IP stacks.
This commit refactors the OS startup functions as follows:
- Moved the OS/app startup functions listed below to "app_startup.c". Their
implementations are now common to all ports (RISC-V and Xtensa) of all
FreeRTOS implementations (IDF and Amazon SMP).
- esp_startup_start_app()
- esp_startup_start_app_other_cores()
- Removed esp_startup_start_app_common() as app startup functions are now
already common to all ports.
- Added extra logs to "main_task" to help with user debugging
Note: Increased startup delay on "unity_task". The "unity_run_menu()" is non
blocking, thus if the main task or other startup tasks have not been freed
by the time "unity_run_menu()" is run, those tasks will be freed the next time
"unity_task" blocks. This could cause some tests to have a memory leak, thus
the "unity_task" startup delay has increased.
This fixes an attempted fix for diram size calculation where it was counted twice, however the fix did not account for cases where iram was not fully filled with cache and therefore was of non 0 size.
Now the calculation should be correct regardless of the cache size.
Closes https://github.com/espressif/esp-idf/issues/9960
A missing flush in the serial reader implementation for
Linux target was causing input to idf.py monitor to not
be forwarded to the application. This is fixed now.
On xtensa architecture, the call to __assert_func uses a reference to __func__ that can
sometimes be placed in flash. Since the __asert_func can be called from functions in IRAM
the check_callgraph script can report an error when checking for invalid calls from IRAM
to flash sections. However, the __asert_func prevents this scenario at runtime so the
check_callgraph script reports a 'flas positive' situation. For this reasson, all references
to __func__$x found prior to a call to __assert_func are droped in the parsing of the rtl files.
Checking that the host is macOS is not sufficient here, since the
linker is still a GNU linker when cross-compiling for a chip.
Instead, use the linker_type variable introduced in the previous
commit.
When compiling for a chip target with Clang,
CMAKE_C_COMPILER_ID="Clang" but the linker is still a GNU linker.
Therefore we can still generate the map file.
When a public header contains _Static_assert or static_assert, check_public_headers.py script will detect it and report it as an issue.
Indeed, public headers shall now use ESP_STATIC_ASSERT.
Host tests of nvs_flash eligible to run in Linux implementation of nvs flash were migrated. Remaining test cases
were left in original folder. Migrated test cases use CMake instead of make.
This commit removes all kconfig_new files and references to them and adds esp-idf-kconfig as a dependency and adequate wrappers to avoid breaking changes.
- Move {target}/rom/miniz.h to common miniz.h
- Add ESP_ROM_HAS_MZ_CRC32 for ESP32/S2/S3/C3/H4
- Alias mz_crc32 to crc32_le if chips not support ESP_ROM_HAS_MZ_CRC32
Closes https://github.com/espressif/esp-idf/issues/10177
1. Fix deep sleep wakeup IOs can not be unhold issue
2. Correct hold related APIs' description
3. Fix gpio_force_hold_all API
docs: Add GPIO wakeup source to sleep_modes doc for ESP32C3 and C2
All the partition handling API functions and data-types were moved from the 'spi_flash' component to the new one named 'esp_partition'. See Storage 5.x migration guide for more details
This commit adds the sdkconfig files for the FreeRTOS test app. These
configurations were dervied from the various legacy unit test app's
config files that included the FreeRTOS component.
This commit tries to keep a 1 to 1 config parity with the legacy test app.
Meaning, if FreeRTOS test were run on a particular target with a particular
config, that config will be represented in one of the test app's
sdkconfig.ci.XXX files.
However, the following configurations were removed for FreeRTOS tests:
- The "freertos_flash" option was removed due to redudancy (already tested in
freertos_options)
pytest_panic used to do 'from conftest import PanicTestDut'. This
stopped working when another conftest.py file was added in
tools/test_build_system/conftest.py.
In this case we can't rename conftest.py since both are necessary
for pytest to install the hooks in the respective test cases.
Fix by moving PanicTestDut into a separate module, then importing
it from there.
The Python dependency checker called from the export scripts and before
build remains offline, i.e. it will use the previously downloaded
constraint file but won't download a newer version.
Related to https://github.com/espressif/esp-idf/pull/9328
* Added port layer from the FreeRTOS POSIX port, added
additional port code for ESP-IDF.
* Created another hello world example using that POSIX
port in tools/test_apps.
* Removed old linux app
This commit ads the corresponding --output-file option from idf_size.py to idf.py.
This is needed because piping the idf.py size output to a file would result in CMake output being present in the file as well.
Unlike COMPILE_OPTIONS, COMPILE_DEFINITIONS CMake property assumes
values without the -D prefix, such as NAME or NAME=VAL.
Previously, IDF build system was passing COMPILE_DEFINITIONS build
property to CMake COMPILE_OPTIONS property, so -D prefix was not
a problem.
Now that COMPILE_DEFINITIONS CMake property is used, -D prefix has
to be removed.
(Note that this doesn't affect 'target_compile_definitions' function,
which strips -D prefix before adding the definition to the property.)
Now that the supported CMake version is >=3.16, this code can be
simplified.
The code to deduplicate the directories can be removed since this is
handled by target_link_directories.
Tools: Support ESP-IDF installed in system-wide shared directory for all users
Closes IDFGH-7790 and IDFGH-7791
See merge request espressif/esp-idf!19479
Test cases calling test_case_uses_tcpip() set critical lwip leak level to 4095, but do not reset it to default.
Hence, 4095 used to get added while calculating the critical threshold for the testcases following them.
This commit removes the riscv_interrupts.h header is it has become redundant. The previously
exposed API has been handled as follows:
- "riscv_interrupt_enable()" and "riscv_interrupt_disable()" have been removed. These functions
were declarations only and never had any implementation.
- "riscv_global_interrupts_enable()" and "riscv_global_interrupts_disable()" renamed to
"rv_utils_intr_global_enable()" and "rv_utils_intr_global_disable()" respectively and now
placed in rv_utils.h
ESP32-C2 has a single group timer, thus it will use it for the interrupt watchdog,
which is more critical than the task watchdog. The latter is implement in
software thanks to the `esp_timer`component.
dl.espressif.com is now using the same root certificate as github.com.
This commit replaces the previously-used ISRG X1 root certificate
with the DigiCert Root CA certificate.
As a result, even if the certificates are not installed (as it happens
on macOS with python.org installers, if the user forgets to run
'Install Certificates.command'), the download is successful.
Related to https://github.com/espressif/esp-idf/issues/4081
This commit fixes build issue because of function `bootloader_common_reset_rtc_retain_mem`
getting inlined with compiler optimization level set to `PERF` (-O2).
Build failure log:
-----------------
In function 'bootloader_common_reset_rtc_retain_mem',
inlined from 'bootloader_common_update_rtc_retain_mem' at /h/esp-idf/components/bootloader_support/src/bootloader_common_loader.c:183:13:
/h/esp-idf/components/bootloader_support/src/bootloader_common_loader.c:159:5: error: 'memset' writing 16 bytes into a region of size 0 overflows the destination [-Werror=stringop-overflow=]
159 | memset(rtc_retain_mem, 0, sizeof(rtc_retain_mem_t));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
update gpio_sig at `spics_out` array in each spi_periph.c of chips later than s2
then `spi_bus_add_device` can correctly distribute gpio_signals for cs_signal
Closes https://github.com/espressif/esp-idf/issues/8876
In previous gpio default isr, interrupt status bits get cleared at the exit of the isr.
However, for edge-triggered interrupt type, the interrupt status bit should be cleared before entering the per-pin handlers to avoid any potential interrupt lost.
Closes https://github.com/espressif/esp-idf/pull/6853
The actual output from the build tool (CMake/Ninja) may or may not
contain color escape codes, depending on various factors. The output
written to the log file should never include color escape codes,
though. This is because color escape codes in files are usually not
rendered as "color" in editors, and complicate reading. Also escape
codes would break the regular expressions used to display hints for
compilation errors.
If stdout is a TTY (meaning that the output is not redirected), tell
the build tool (GNU Make or Ninja) to enable colorized output.
GNU Make and Ninja also check if their stdout is redirected and
strip color escape sequences in that case. CLICOLOR_FORCE environment
variable overrides this behavior.
With this change, if the compiler was launched with the
-fcolor-diagnostics flag and idf.py output is not redirected, the
final output in the terminal will be colorized.
(-fcolor-diagnostics is handled at CMake level by the previous commit)
Related to https://github.com/espressif/esp-idf/issues/4162
Setting this option informs CMake that it should pass
-fcolor-diagnostics flag to the compiler.
(Colorized build system output, like from GNU Make, is produced even
without this flag.)
Note that if the build is done using Ninja and the build output is
redirected (not a TTY), Ninja will still strip the escape codes from
the output. For the case of idf.py, this is handled in the next
commit.
There are multiple changes in this commit:
1. Unify the RISC-V and ULP-FSM code paths in esp32ulp_mapgen.py.
It seems that these were originally introduced because `nm` output
for the RISC-V case contained symbol sizes, while for the ULP-FSM
no symbol sizes were reported. This makes sense, because the
ULP-FSM object files are produced from assembly source, symbol
sizes have to be added manually using the .size directive.
In the case of RISC-V, the object files are built from C sources
and the sizes are automatically added by the compiler.
Now 'posix' output format is used for both RISC-V and ULP-FSM.
2. Move BASE_ADDR out of esp32ulp_mapgen.py. This now has to be passed
from CMake, which should make it easier to modify if a new chip
with a different RTC RAM base address is added.
3. Add C++ guards to the generated header file.
4. Switch from optparse to argparse for similarity with other IDF
tools.
5. Add type annotations.
This fixes the issue with build output not being colorized on Windows,
while the hints messages are colorized.
The issue occurred because sys.stdout and sys.stderr get overridden
by colorama.init() at runtime, but the default argument
output_stream=sys.stdout holds the reference to the"original"
sys.stdout.
colorama.init() (which, by the way, gets called via a curious chain
of imports, via idf_component_tools.manifest and tqdm package)
overrides standard streams, on Windows only. The overridden streams
contain logic to convert ANSI color codes into Windows Console API
calls to colorize the text.
Since read_and_write_stream function used the default value of
output_stream evaluated at module loading time, it was using the
original sys.stdout, not the one overridden by colorama.
One extra note is that while this does fix the coloring issue, the
solution is a bit fragile, as it relies on one of the following
(on Windows):
- colorama.init() is called (this can change if idf-component-manager
stops importing tqdm)
- Sufficiently new version of Windows 10 is used, and ANSI color codes
support is enabled in the Registry.
The actual output from the build tool (CMake/Ninja) may or may not
contain color escape codes, depending on various factors. The output
written to the log file should never include color escape codes,
though. This is because color escape codes in files are usually not
rendered as "color" in editors, and complicate reading. Also escape
codes would break the regular expressions used to display hints for
compilation errors.
If stdout is a TTY (meaning that the output is not redirected), tell
the build tool (GNU Make or Ninja) to enable colorized output.
GNU Make and Ninja also check if their stdout is redirected and
strip color escape sequences in that case. CLICOLOR_FORCE environment
variable overrides this behavior.
With this change, if the compiler was launched with the
-fcolor-diagnostics flag and idf.py output is not redirected, the
final output in the terminal will be colorized.
(-fcolor-diagnostics is handled at CMake level by the previous commit)
Related to https://github.com/espressif/esp-idf/issues/4162
Setting this option informs CMake that it should pass
-fcolor-diagnostics flag to the compiler.
(Colorized build system output, like from GNU Make, is produced even
without this flag.)
Note that if the build is done using Ninja and the build output is
redirected (not a TTY), Ninja will still strip the escape codes from
the output. For the case of idf.py, this is handled in the next
commit.
- primary reason: 4.6.2 fixes errors if the user's home directory contains a space
- alternative fix: the CCACHE_DIR env var can be set to something without a space in it, or set to TMP which properly escapes the home dir name
- there may be other issues with spaces in filenames on builds in other parts of ESP-IDF, see https://github.com/espressif/esp-idf/issues/8364 for more info
tested locally on windows only, use at your own risk
This adds CSV support to idf_size.py and idf.py size actions and using the --format argument which accepts 'text', 'json' or 'csv' as input.
idf_size.py --json argument is deprecated but left to avoid a breaking change.
For idf.py size actions OUTPUT_JSON environment variable set at configuration time is overriden at target build time if --format is used.
Additionally, this commit refactors big parts of code, unified usage of json_dict and manually generated dictionaries for textual output and improves code quality in many parts.
On the build runners, the installation frequently takes around 150
seconds, which is above the current timeout. This change increases
the timeout.
Closes IDFCI-1436
This commit fixes gpio_hold_en(pin) function for ESP32, where after wakeup from deep sleep, the pin gets reset to default state and stop holding the pin level.
'linux' component provides some of the common header files, such as
'sys/queue.h' and 'sys/lock.h'. For chip targets, it is possible to
include these files without having to add any extra requirements.
With this change, the same behavior will apply for the linux target.
libpython2.7 was added to the container to allow running GDB built
with Python 2.7 support and distributed as part of the cross-compiler
toolchain.
Now that we have a new release of GDB which works with Python 3.x,
the GDB shipped with the cross-compiler is no longer used. Removing
libpython2.7 should reduce the image size.
This reverts commit be0372b1db.
uart: update console docs about frequency for ESP32-C2, move frequency of clock sources out of HAL
Closes IDF-5424 and IDF-4332
See merge request espressif/esp-idf!19274
sizeof(time_t) was previously switched from 4 to 8, ROM functions that use
time_t or dependent types (such as "struct stat") are no longer called due as
they still treat sizeof(time_t) as 4 (see commit
24c20d188e).
However, there is a ROM callpath that was left out. If putchar is the first
stdio print related call, the call path will result in cantwrite() ->
__swsetup_r() -> __smakebuf_r() -> __swhatbuf_r() using the ROM "struct stat"
(where sizeof(time_t)==4).
Instead of removing all printf related ROM newlib functions (which will result
in increased binary size), this commit adds a workaround to setup the stdio
files before any print related calls occur.
This results in cantwrite() always returning false, thus the callpath described
above never being reached.
Closes https://github.com/espressif/esp-idf/issues/9269
hints are added:
- to indicate that esp_cpu_ccount_t must be replaced by esp_cpu_cycle_count_t
- to indicate that esp_cpu_get/set_ccount must be replaced by esp_cpu_get/set_cycle_count
- to indicate that soc/cpu.h and compare_set.h were removed and user must include esp_cpu.h instead
- to indicate that esp_intr.h was removed and user must include esp_intr_alloc.h instead
- to indicate that esp_panic.h was made private and user must use the functionalities from esp_debug_helper.h instead
- to indicate that spilock.h, clk_ctrl_os.h and rtc_wdt.h must be included without the soc/
- to indicate that soc_log.h was renamed esp_hw_log.h and made private
`__COMPONENT_TARGETS` is evaluated very early when components and
component directories are added to the build, which means that all
components (including the ones which are in EXCLUDE_COMPONENTS) have
a build system target defined. The component manager was given the
list of all known components (derived from the list of targets), not
the list of components after EXCLUDE_COMPONENTS were processed.
Because of that, EXCLUDE_COMPONENTS didn't effectively exclude the
component from the consideration of the component manager.
Since both chips have built-in JTAG functionality, and there are no
official boards with FT2232H for these chips, use the built-in JTAG
by default.
To use them with esp-prog, set:
OPENOCD_COMMANDS="-f board/esp32c3-ftdi.cfg"
or pass this via the --openocd-commands argument to idf.py.
hardware issue: we can't control the IDLE level by the stop item
when loop transmission is enabled.
But we can always control the IDLE state by register.
As the tlsf implementation is a fork from https://github.com/mattconte/tlsf,
the sources are moved to a separate repository and used as a submodule in the esp-idf instead.
In this commit:
- Removing TLSF related files and using tlsf submodule instead.
- Adding components/heap/tlsf_platform.h header gathering all IDF specifics.
- The multi_heap_poisoning.c provides the declaration of the
function block_absorb_post_hook() definied weak in the TLSF repository.
- The tlsf_platform.h includes the tlsf_common.h file after the definition
of FL_INDEX_MAX_PLATFORM macro to make sure that this macro will be available
in tlsf_common.h without having to include tlaf_platform.h from IDF in the
tlsf_common.h header from the TLSF repository.
- Add missing include from tlsf_block_functions.h in the multi_heap.c file.
Change related to the changes made in TLSF repository (tlsf_block_functions.h
no longer included in tlsf.h)
Blocking read from cdcacm VFS could return less bytes than requested.
This didn’t match the behaviour of other VFS drivers, and higher level
code could misbehave.
This commit marks all functions in soc_hal.h and soc_ll.h as deprecated.
Users should use functions from esp_cpu.h instead.
Also added missing wrap funcions for esp_cpu_stall() in test_panic.c files.
This commit marks all functions in interrupt_controller_hal.h, cpu_ll.h and cpu_hal.h as deprecated.
Users should use functions from esp_cpu.h instead.
Closes: https://github.com/espressif/esp-idf/issues/9208
When I2S is configured into different modes, the slot sequence varies.
This commit updates slot sequence tables and corresponding descriptions
in (both code and programming guide).
Fix dependency tree so that lwip doesn't depend on any specific network
interface component.
Network interface drivers shall depend on esp_netif.
esp_netif shall depend on lwip (but not on any specific interface
driver) -- it optionally depends on vfs and esp_eth (need ethernet
header for L2/bridge mode)
If self.elf_exists is false, self.gdb_helper is None, which caused this
code to crash. It looks like this guard was missed in e30329ffe2.
Previously the code could crash like this:
Traceback (most recent call last):
File "/usr/local/src/esp-idf/tools/idf_monitor.py", line 376, in <module>
main()
File "/usr/local/src/esp-idf/tools/idf_monitor.py", line 367, in main
monitor.main_loop()
File "/usr/local/src/esp-idf/tools/idf_monitor.py", line 163, in main_loop
self._main_loop()
File "/usr/local/src/esp-idf/tools/idf_monitor.py", line 268, in _main_loop
super()._main_loop()
File "/usr/local/src/esp-idf/tools/idf_monitor.py", line 203, in _main_loop
self.serial_handler.handle_commands(data, self.target, self.run_make, self.console_reader,
File "/usr/local/src/esp-idf/tools/idf_monitor_base/serial_handler.py", line 203, in handle_commands
run_make_func('encrypted-flash' if self.encrypted else 'flash')
File "/usr/local/src/esp-idf/tools/idf_monitor.py", line 150, in run_make
run_make(target, self.make, self.console, self.console_parser, self.event_queue, self.cmd_queue,
File "/usr/local/src/esp-idf/tools/idf_monitor.py", line 237, in __exit__
self.serial_reader.gdb_exit = self.gdb_helper.gdb_exit # write gdb_exit flag
AttributeError: 'NoneType' object has no attribute 'gdb_exit'
GPIO_HOLD_MASK array was missing the last item
Add __Static_assert to check array sizes for all gpio_periph.c files to prevent same mistake in the future.
All gpio hal and ll functions input arguments gpio_num_t are changed to uint32_t type.
Validation of gpio num should be guaranteed from the driver layer.
feat: new find_build_apps rules with centralized manifest file
Closes IDFCI-1061, IDF-3553, IDFCI-941, IDFCI-1359, and IDFCI-1361
See merge request espressif/esp-idf!18299
Constraint files can be disabled with environment variable as well which
is useful when one uses the install/export scripts instead of
idf_tools.py directly. This is option is useful for offline build as
well.
Closes https://github.com/espressif/esp-idf/issues/9263
* Added C++ sentinels if missed
* Used #pragma once, removed macro's with leading underscores
* Updated copyright checker to allow "BSD-2-Clause-FreeBSD AND
Apache-2.0" for newlib files
* Fixed minor compilation issues/implicit inclusions
The following two functions in bootloader_support are private now:
* esp_secure_boot_verify_sbv2_signature_block()
* esp_secure_boot_verify_rsa_signature_block()
They have been moved into private header files
inside bootloader_private/
* Removed bootloader_reset_reason.h and
bootloader_common_get_reset_reason() completely.
Alternative in ROM component is available.
* made esp_efuse.h independent of target-specific rom header
Previously psram driver was in esp_hw_support, which is a common
requirement(when in non-bootloader build). So when building
idf_as_lib (on esp32), the mfix-esp32-psram-cache-issue flag
will always be added (when CONFIG_SPIRAM_CACHE_WORKAROUND is on).
As psram driver is moved to esp_psram component, and esp_psram
is added via idf_component_optional_requires, which is a "weak"
dependency. You need to include esp_psram compoennt explicitly
if psram will be in use.
As idf_as_lib doesn't need psram, this test can be removed.
When ESP32-C2 is paired with a 26 MHz XTAL, the systimer tick
frequency becomes equal to 26 / 2.5 = 10.4 MHz. Previously we always
assumed that systimer tick frequency is integer (and 1 MHz * power of
two, above that!).
This commit introduces a new LL macro, SYSTIMER_LL_TICKS_PER_US_DIV.
It should be set in such a way that:
1. SYSTIMER_LL_TICKS_PER_US / SYSTIMER_LL_TICKS_PER_US_DIV equals the
actual systimer tick frequency,
2. and SYSTIMER_LL_TICKS_PER_US is integer.
For ESP32-C2 this means that SYSTIMER_LL_TICKS_PER_US = 52 and
SYSTIMER_LL_TICKS_PER_US_DIV = 5.
This introduced two possible issues:
1. Overflow when multiplying systimer counter by 5
- Should not be an issue, since systimer counter is 52-bit, so
counter * 5 is no more than 55-bit.
2. The code needs to perform:
- divide by 5: when converting from microseconds to ticks
- divide by 52: when converting from ticks to microseconds
The latter potentially introduces a performance issue for the
esp_timer_get_time function.
esp-ringbuf funtion placement is now controlled by its own configs:
CONFIG_RINGBUF_PLACE_FUNCTIONS_INTO_FLASH and CONFIG_RINGBUF_PLACE_ISR_FUNCTIONS_INTO_FLASH
Closes https://github.com/espressif/esp-idf/issues/9198
