Regression in ede477ea65 for host tests only - ESP-IDF supports
strlcpy() but strlcpy & strlcat are currently no-ops in the host
tests (to avoid libbsd dependency).
Additionally, always enable the partition MD5 check if flash encryption is on in
Release mode. This ensures the partition table ciphertext has not been modified
(CVE-2021-27926).
The exception is pre-V3.1 ESP-IDF bootloaders and partition tables, which
don't have support for the MD5 entry.
spi_flash: refactoring flash encryption implementations into new api
Closes IDFGH-4493, IDFGH-4423, and IDF-657
See merge request espressif/esp-idf!12569
When `DIS_USB_JTAG` eFuse is NOT burned (`False`), it is not possible
to set pins 18 and 19 as GPIOs. This commit solves this by manually
disabling USB JTAG when using pins 18 or 19.
The functions shall use `gpio_hal_iomux_func_sel` instead of
`PIN_FUNC_SELELECT`.
Enable shared stack watchpoint for overflow detection
Enable unit tests:
* "test printf using shared buffer stack" for C3
* "Test vTaskDelayUntil" for S2
* "UART can do poll()" for C3
On C3 the cache is programmatically split between Icache and dcache and with the default setup we dont leave a lot pages
available for additional mmaps into instruction space. Disabling this test for now since any hypothetical use case for this
is no longer supported "out of the box"
This reverts commit bf35ef1ce7.
It has been noticed that there are scenarios where even though firmware is not enabled
with flash encryption config feature, it should be able to write to encrypted partitions.
This revert adds the feature back which was removed around v4.0 timelines, and same
change will be backported to all releases (upto v4.0) for consistency.
- Adds arbitration between the erase and write in suspend mode
(If the flash memory is suspended then only a read operation can be performed.)
- espcoredump: Dump does not use suspend feature, just resume before continue.
- spi_flash: Add release_cpu() to do suspend/release_cpu/resume
There is no ccomp timer on C3, which means our performance tests will start
failing again due to variance caused by cache misses.
This MR adds TEST_PERFORMANCE_CCOMP_ macro that will only fail
performance test if CCOMP timer is supported on the target
* General tests like page loading from flash
* Rough test of fixed-size data types
* Rough test of blob read
* Added coverage target in cmake, also accessible
via `idf.py coverage`
* Fixed unsigned comparison in comp. enum table
* introducing temporary LINUX_TARGET define
The following three headers will be mockes:
* esp_flash.h
* esp_spi_flash.h
* esp_partition.h
* counter functions live in own header
* add spi_flash sim dir for esp_err.h to Unity
* modified gen_esp_err_to_name.py to ignore
sim/ dir in spi_flash component
Add cmock .yaml config file
Add spi hal header until soc can mock the hal
layer as well.
* add toolchain file
* add linux to preview targets
* add stub for dfu number in cmake
* excluded unity runner per default
* Added esp_attr.h and esp_partition.h linux stubs
* component.cmake check list for emptyness
* added switch for linux in unity cmake file
* Added Linux host example app
From now on, we have two tags for esp_flash tests:
- [esp_flash] for main flash chip only tests
- [esp_flash_3] for tests with external flash chips
To Run all tests, type `[esp_flash`; to run tests for main flash chip
only, type `[esp_flash].
There is a periodically yield in the esp_flash driver, to ensure the
cache will not be disabled for too long on ESP32.
On ESP32-S2 and later, we need to support more different kind of yield:
1. polling conditions, including timeout, SW read request, etc.
2. wait for events, including HW done/error/auto-suspend, timeout
semaphore, etc.
The check_yield() and yield() is separated into two parts, because we
may need to insert suspend, etc. between them.
1. The 2nd bootloader always call `rom_spiflash_unlock()`, but never help to clear the WEL bit when exit. This may cause system unstability.
This commit helps to clear WEL when flash configuration is done.
**RISK:** When the app starts, it didn't have to clear the WEL before it actually write/erase. But now the very first write/erase operation should be done after a WEL clear. Though the risk is little (all the following write/erase also need to clear the WEL), we still have to test this carefully, especially for those functions used by the OTA.
2. The `rom_spiflash_unlock()` function in the patch of ESP32 may (1) trigger the QPI, (2) clear the QE or (3) fail to unlock the ISSI chips.
Status register bitmap of ISSI chip and GD chip:
| SR | ISSI | GD25LQ32C |
| -- | ---- | --------- |
| 0 | WIP | WIP |
| 1 | WEL | WEL |
| 2 | BP0 | BP0 |
| 3 | BP1 | BP1 |
| 4 | BP2 | BP2 |
| 5 | BP3 | BP3 |
| 6 | QE | BP4 |
| 7 | SRWD | SRP0 |
| 8 | | SRP1 |
| 9 | | QE |
| 10 | | SUS2 |
| 11 | | LB1 |
| 12 | | LB2 |
| 13 | | LB3 |
| 14 | | CMP |
| 15 | | SUS1 |
QE bit of other chips are at the bit 9 of the status register (i.e. bit 1 of SR2), which should be read by RDSR2 command.
However, the RDSR2 (35H, Read Status 2) command for chip of other vendors happens to be the QIOEN (Enter QPI mode) command of ISSI chips. When the `rom_spiflash_unlock()` function trys to read SR2, it may trigger the QPI of ISSI chips.
Moreover, when `rom_spiflash_unlock()` try to clear the BP4 bit in the status register, QE (bit 6) of ISSI chip may be cleared by accident. Or if the ISSI chip doesn't accept WRSR command with argument of two bytes (since it only have status register of one byte), it may fail to clear the other protect bits (BP0~BP3) as expected.
This commit makes the `rom_spiflash_unlock()` check whether the vendor is issi. if so, `rom_spiflash_unlock()` only send RDSR to read the status register, send WRSR with only 1 byte argument, and also avoid clearing the QE bit (bit 6).
3. `rom_spiflash_unlock()` always send WRSR command to clear protection bits even when there is no protection bit active. And the execution of clearing status registers, which takes about 700us, will also happen even when there's no bits cleared.
This commit skips the clearing of status register if there is no protection bits active.
Also move the execute_flash_command to be a bootloader API; move
implementation of spi_flash_wrap_set to the bootloader
* partition api changed from spi_flash* API to
esp_partition* API and is abstracted as a C++
interface.
* The old nvs encryption is still possible
* changed default unit test app partition table
* Partitions coming from esp_partition API are
checked for generic flash encryption. If yes,
an error is returned since generic flash
encryption isn't compatible with nvs
encryption
* esp32, esp32s2 tests don't require nvs_flash
but mbedtls now
Closes IDF-1340
Closes IDF-858
* changing dependencies from unity->cmock
* added component.mk and Makefile.projbuild
* ignore test dir in gen_esp_err_to_name.py
* added some brief introduction of CMock in IDF
Sometimes the flash size read from bootloader is not correct. This may
forbid SPI Flash driver from reading the the area larger than the size
in bootloader header.
When the new config option is enabled, the latest configured
ESPTOOLPY_FLAHSIZE in the app header will be used to override the value
read from bootloader header.
Including:
1. Change the write bytes/read bytes parameter in the host driver into slicers to meet the requirements of complicated cases.
2. Refactor the esp_flash_api code a bit so that we can use the code in the ROM laster
3. Provide get_temp_buffer and release_temp_buffer in the os_functions when the buffer passed by application cannot be used directly.
4. Make timeout of operations configurable in the chip_driver.
5. Make dummy number configurable.
In commit 309376f51a, it seems like regression
was added to use ROM level API for disabling flash write protection. This
started random firmware crashes (on specific modules) with exception
`IllegalInstruction` during encrypted flash writes.
Fix here removes relevant ROM API call, since disabling flash write protection
is already ensured by caller of this API.
Closes https://github.com/espressif/esp-idf/issues/5467
Flash write operation is broken down into smaller chunk writes. Size
of this chunk was previously set to 8K but that in-turn meant cache and
non-IRAM resident interrupts could stay disabled upto ~24msec for 8K flash
write operation. If chunk size is brought down to 256 (typical flash page size)
then it brings down cache and non-IRAM interrupts disable duration to ~1msec.
Fix here keeps defaults same but provides configuration option to tweak the
setting based on application requirement.
The SPI bus lock on SPI1 introduces two side effects:
1. The device lock for the main flash requires the
`CONFIG_FREERTOS_SUPPORT_STATIC_ALLOCATION` to be selected, however this
option is disabled by default in earlier IDF versions. Some developers
may find their project cannot be built by their old sdkconfig files.
2. Usually we don't need the lock on the SPI1 bus, due to it's
restrictions. However the overhead still exists in this case, the IRAM
cost for static version of semaphore functions, and the time cost when
getting and releasing the lock.
This commit:
1. Add a CONFIG_SPI_FLASH_BYPASS_MAIN_LOCK option, which will forbid the
space cost, as well as the initialization of the main bus lock.
2. When the option is not selected, the bus lock is used, the
`CONFIG_FREERTOS_SUPPORT_STATIC_ALLOCATION` will be selected explicitly.
3. Revert default value of `CONFIG_FREERTOS_SUPPORT_STATIC_ALLOCATION`
to `n`.
introduced in 49a48644e4.
Closes https://github.com/espressif/esp-idf/issues/5046
We used to manually specify the CS id. However after the SPI bus lock is
introduced, the lock is responsible to assign the CS lines and provide
the CS id. The esp_flash driver now depends on the ID assigned by the
SPI bus lock, the configuration field is deprecated.
The issue is introduced in 571864e8ae. The
esp_flash API tries to clear the QE bit when the flash is not working in
quad modes.
However this introduces a regression, compared to earlier versions and
the legacy API. When the chip is not detected, the generic chip driver
is used, which cannot 100% handle the QE bit properly for all flash
vendors. There may be some flash chips (e.g. MXIC) that can be used in
dual modes by legacy API, but output wrong data when the esp_flash API
clears the QE bit in a wrong way.
This commit reverts the QE force clearing behavior, so that it's safer
for the generic chip driver to work under dual modes.
The default chip driver (chip_generic) use command 01H + 2 bytes to
clear the QE bit. However this will accidently change the configuration
register value of the MXIC chip.
MXIC chip driver is added to fix that.
Allocation of the temporary internal buffer will now repeat until a small enough buffer can be
allocated, and only fail if less than a 256 byte block of internal RAM is free.
Adds unit test for the same, and generic test utility for creating memory pressure.
Previously would try allocate buffer of minimum size 16KB not maximum size 16KB, causing
out of memory errors for any large reads, or if less than 16KB contiguous free heap.
Also, if using legacy API and internal allocation failed then implementation would abort()
instead of returning the error to the caller.
Added test for using large buffers in PSRAM.
Closes https://github.com/espressif/esp-idf/issues/4769
Also reported on forum: https://esp32.com/viewtopic.php?f=13&t=14304&p=55972
DISABLED_FOR_TARGETS macros are used
Partly revert "ci: disable unavailable tests for esp32s2beta"
This partly reverts commit 76a3a5fb48.
Partly revert "ci: disable UTs for esp32s2beta without runners"
This partly reverts commit eb158e9a22.
Partly revert "fix unit test and examples for s2beta"
This partly reverts commit 9baa7826be.
Partly revert "efuse: Add support for esp32s2beta"
This partly reverts commit db84ba868c.
1. add hal and low-level layer for timer group
2. add callback functions to handle interrupt
3. add timer deinit function
4. add timer spinlock take function
Tests for external flash chips used to controlled by macros, one bin for
one chip. And tests are done manually. This commit refactored the test
so that all 3 chips can all run in single test.
There used to be dummy phase before out phase in common command
transactions. This corrupts the data.
The code before never actually operate (clear) the QE bit, once it finds
the QE bit is set. It's hard to check whether the QE set/disable
functions work well.
This commit:
1. Cancel the dummy phase
2. Set and clear the QE bit according to chip settings, allowing tests
for QE bits. However for some chips (Winbond for example), it's not
forced to clear the QE bit if not able to.
3. Also refactor to allow chip_generic and other chips to share the same
code to read and write qe bit; let common command and read command share
configure_host_io_mode.
4. Rename read mode to io mode since maybe we will write data with quad
mode one day.
esp_partition_register_external did not call load_partitions, so if
it was called before any call to esp_partition_find, then the main
partition table would never be loaded. Introduce new function,
ensure_partitions_loaded, and call it both from esp_partition_find and
esp_partition_register_external.
Closes https://github.com/espressif/esp-idf/issues/4116
1. The translation should be first reviewed by technical reviewers;
2. and then by language reviewers.
For the translation for the first batch of files, please see !MR5620 and !MR5613
During coredump, dangerous-area-checking should be disabled, and cache
disabling should be replaced by a safer version.
Dangerous-area-checking used to be in the HAL, but it seems to be more
fit to os functions. So it's moved to os functions. Interfaces are
provided to switch between os functions during coredump.
When legacy mode is used, the coredump still fails during linking
because "esp_flash_init_default_chip", "esp_flash_app_init" and
"esp_flash_default_chip " are not compiled and linked.
Instead of using ``if`` macros in callers, these functions are protected
by ``if`` macros in the header, and also not compiled in the sources.
"esp_flash_default_chip" variable is compiled with safe default value.
spi_flash has been updated and its functions work from flash by default instead of IRAM that cause issue
add Kconfig value into espcoredump to enable spi_flash legacy mode (CONFIG_SPI_FLASH_USE_LEGACY_IMPL) when core dump is selected
fix spi_flash issues to work correctly with legacy mode when CONFIG_SPI_FLASH_USE_LEGACY_IMPL is used
Add support for get write protection support, fixed the duplicated
set_write_protection link.
All the write_protection check in the top layer are removed. The lower
levels (chip) should ensure to disable write protection before the
operation start.
The name "start_addr" (which goes straight into the docs) implies
it's an absolute address while in fact it's an offset into the
partition like what's used in all the other esp_partition_*
functions.
So in order to avoid confusion make the name consistent with the
parameter names used for the other partition functions and call it
"offset".
Merges https://github.com/espressif/esp-idf/pull/3750
Do not include bootloader in flash target when secure boot is enabled.
Emit signing warning on all cases where signed apps are enabled (secure
boot and signed images)
Follow convention of capital letters for SECURE_BOOT_SIGNING_KEY
variable, since it is
relevant to other components, not just bootloader.
Pass signing key and verification key via config, not requiring
bootloader to know parent app dir.
Misc. variables name corrections
!4452 and !4897 made some ill-advised corrections to dependency info;
revert those in this MR. Handling pre-built binaries as imported
libraries is retained, however.
Using xxx_periph.h in whole IDF instead of xxx_reg.h, xxx_struct.h, xxx_channel.h ... .
Cleaned up header files from unnecessary headers (releated to soc/... headers).
When flash work in DIO Mode, in order to ensure the fast read mode of flash
is a fixed value, we merged the mode bits into address part, and the fast
read mode value is 0 (the default value).
According to the documentation[1][2] for partitions, setting the encrypted
flag for partitions should be a no-op when system level encryption isn't
enabled. The current implementation, however, does not actually match
the documentation and it ends up with an unreadable partition via the
partition API if a partition flag is marked as encrypted without
system-level encryption enabled. (This is because the writes go through
the encryption block, and reads do not go through the encryption block
when this situation occurs causing unreadable data to the application
running.) This fixes up the read-back of the partition table to match
whether or not the partition is currently encrypted under the hood.
This should not affect the bootloader's code for reading/writing encrypted
partitions as the bootloader directly invokes the spi_flash_write*(...)
APIs.
[1] https://docs.espressif.com/projects/esp-idf/en/latest/api-guides/partition-tables.html#flags
[2] https://docs.espressif.com/projects/esp-idf/en/latest/security/flash-encryption.html#encrypted-partition-flag
Closes https://github.com/espressif/esp-idf/pull/3328
Signed-off-by: Tim Nordell <tim.nordell@nimbelink.com>
This MR removes the common dependency from every IDF components to the SOC component.
Currently, in the ``idf_functions.cmake`` script, we include the header path of SOC component by default for all components.
But for better code organization (or maybe also benifits to the compiling speed), we may remove the dependency to SOC components for most components except the driver and kernel related components.
In CMAKE, we have two kinds of header visibilities (set by include path visibility):
(Assume component A --(depends on)--> B, B is the current component)
1. public (``COMPONENT_ADD_INCLUDEDIRS``): means this path is visible to other depending components (A) (visible to A and B)
2. private (``COMPONENT_PRIV_INCLUDEDIRS``): means this path is only visible to source files inside the component (visible to B only)
and we have two kinds of depending ways:
(Assume component A --(depends on)--> B --(depends on)--> C, B is the current component)
1. public (```COMPONENT_REQUIRES```): means B can access to public include path of C. All other components rely on you (A) will also be available for the public headers. (visible to A, B)
2. private (``COMPONENT_PRIV_REQUIRES``): means B can access to public include path of C, but don't propagate this relation to other components (A). (visible to B)
1. remove the common requirement in ``idf_functions.cmake``, this makes the SOC components invisible to all other components by default.
2. if a component (for example, DRIVER) really needs the dependency to SOC, add a private dependency to SOC for it.
3. some other components that don't really depends on the SOC may still meet some errors saying "can't find header soc/...", this is because it's depended component (DRIVER) incorrectly include the header of SOC in its public headers. Moving all this kind of #include into source files, or private headers
4. Fix the include requirements for some file which miss sufficient #include directives. (Previously they include some headers by the long long long header include link)
This is a breaking change. Previous code may depends on the long include chain.
You may need to include the following headers for some files after this commit:
- soc/soc.h
- soc/soc_memory_layout.h
- driver/gpio.h
- esp_sleep.h
The major broken include chain includes:
1. esp_system.h no longer includes esp_sleep.h. The latter includes driver/gpio.h and driver/touch_pad.h.
2. ets_sys.h no longer includes soc/soc.h
3. freertos/portmacro.h no longer includes soc/soc_memory_layout.h
some peripheral headers no longer includes their hw related headers, e.g. rom/gpio.h no longer includes soc/gpio_pins.h and soc/gpio_reg.h
BREAKING CHANGE
1. separate rom include files and linkscript to esp_rom
2. modefiy "include rom/xxx.h" to "include esp32/rom/xxx.h"
3. Forward compatible
4. update mqtt
On flash program operation (either erase or write), if corresponding address has
cache mapping present then cache is explicitly flushed (for both pro and app cpu)
Closes https://github.com/espressif/esp-idf/issues/2146
Added:
* set a secure version in app/bootloader.
* description anti-rollback to ota part
* emulate the secure_version write and read operations
* efuse_em partition.
* a description about a rollback for native_ota_example.
Closes: TW26335
New unity component can be used for testing other applications.
Upstream version of Unity is included as a submodule.
Utilities specific to ESP-IDF unit tests (partitions, leak checking
setup/teardown functions, etc) are kept only in unit-test-app.
Kconfig options are added to allow disabling certain Unity features.
Routine `spi_flash_cache2phys` sometimes return incorrect value,
resulting in failure in getting currently running ota partition.
This in turn aborts firmware update process.This issue was more
prominent with SPIRAM enabled cases.
Fix ensures proper cache guards during `spi_flash_cache2phys`, and
also for few other similar APIs.
In addition, `esp_ota_get_running_partition` has also been optimized
to save currently running partition for subsequent invocations.
Fixes https://github.com/espressif/esp-idf/issues/2451
Added bootloader_common_get_sha256_of_partition() and esp_partition_get_sha256() - get or calculate SHA-256
digest for app and data partitions.
Added bootloader_sha256_hex_to_str() - helps to print SHA-256 digest
Added esp_partition_check_identity() - compares two partitions by SHA-256 digest
Refactoring a function esp_image_load() in bootloader space to esp_image_verify() and
bootloader_load_image(). Old name function esp_image_load is deprecated
and will remove in V4.0 version.
spi_flash/sim: Fix error test_host. Add stub for bootloader_common_get_sha256_of_partition in sim/stubs
* spi_flash_mmap_pages needs pages array to be in internal memory.
Document and check this.
* Fix a bug that spi_flash_mmap did not allocate pages array in
internal memory.
* Minor style fixes: const-ify pages argument of spi_flash_mmap, add
spaces around operators, mark output arguments with [out].
Closes https://github.com/espressif/esp-idf/issues/2229.
components/spi_flash/partition.c: In function 'load_partitions':
components/spi_flash/partition.c:179:66: error: argument to 'sizeof' in 'strncpy' call is the same expression as the source; did you mean to use the size of the destination? [-Werror=sizeof-pointer-memaccess]
strncpy(item->info.label, (const char*) it->label, sizeof(it->label));
^
The fix is for the situation when cache disabling mechanism causes
a deadlock with user tasks. Situation is as follows:
1. spi_flash operation is started from low-priority task on CPU0
2. It uses IPC to wake up high-priority IPC1 task on CPU1, preventing
all other tasks on CPU1 from running. This is needed to safely
disable the cache.
3. While the task which started spi_flash operation is waiting for IPC1
task to acknowledge that CPU1 is not using cache anymore, it is
preempted by a higher priority application task ("app0").
4. Task app0 busy-waits for some operation on CPU1 to complete. But
since application tasks are blocked out by IPC1 task, this never
happens. Since app0 is busy-waiting, the task doing spi flash
operation never runs.
The more or less logical soltion to the problem would be to also do
cache disabling on CPU0 and the SPI flash operation itself from IPC0
task. However IPC0 task stack would need to be increased to allow doing
SPI flash operation (and IPC1 stack as well). This would waste some
memory. An alternative approach adopted in this fix is to call FreeRTOS
functions to temporary increase the priority of SPI flash operation task
to the same level as the IPC task.
Fixes https://github.com/espressif/arduino-esp32/issues/740
Fixes https://github.com/espressif/esp-idf/issues/1157
IDF had two defines ESP_PARTITION_TABLE_OFFSET and ESP_PARTITION_TABLE_ADDR. They were the same and equal by 0x8000.
A define ESP_PARTITION_TABLE_ADDR was removed from IDF.
A define ESP_PARTITION_TABLE_OFFSET depends on option CONFIG_PARTITION_TABLE_OFFSET in Kconfig.
Now using only #define ESP_PARTITION_TABLE_OFFSET CONFIG_PARTITION_TABLE_OFFSET
