spi_flash: refactoring flash encryption implementations into new api
Closes IDFGH-4493, IDFGH-4423, and IDF-657
See merge request espressif/esp-idf!12569
When `DIS_USB_JTAG` eFuse is NOT burned (`False`), it is not possible
to set pins 18 and 19 as GPIOs. This commit solves this by manually
disabling USB JTAG when using pins 18 or 19.
The functions shall use `gpio_hal_iomux_func_sel` instead of
`PIN_FUNC_SELELECT`.
Enable shared stack watchpoint for overflow detection
Enable unit tests:
* "test printf using shared buffer stack" for C3
* "Test vTaskDelayUntil" for S2
* "UART can do poll()" for C3
On C3 the cache is programmatically split between Icache and dcache and with the default setup we dont leave a lot pages
available for additional mmaps into instruction space. Disabling this test for now since any hypothetical use case for this
is no longer supported "out of the box"
This reverts commit bf35ef1ce7.
It has been noticed that there are scenarios where even though firmware is not enabled
with flash encryption config feature, it should be able to write to encrypted partitions.
This revert adds the feature back which was removed around v4.0 timelines, and same
change will be backported to all releases (upto v4.0) for consistency.
- Adds arbitration between the erase and write in suspend mode
(If the flash memory is suspended then only a read operation can be performed.)
- espcoredump: Dump does not use suspend feature, just resume before continue.
- spi_flash: Add release_cpu() to do suspend/release_cpu/resume
There is no ccomp timer on C3, which means our performance tests will start
failing again due to variance caused by cache misses.
This MR adds TEST_PERFORMANCE_CCOMP_ macro that will only fail
performance test if CCOMP timer is supported on the target
* General tests like page loading from flash
* Rough test of fixed-size data types
* Rough test of blob read
* Added coverage target in cmake, also accessible
via `idf.py coverage`
* Fixed unsigned comparison in comp. enum table
* introducing temporary LINUX_TARGET define
The following three headers will be mockes:
* esp_flash.h
* esp_spi_flash.h
* esp_partition.h
* counter functions live in own header
* add spi_flash sim dir for esp_err.h to Unity
* modified gen_esp_err_to_name.py to ignore
sim/ dir in spi_flash component
Add cmock .yaml config file
Add spi hal header until soc can mock the hal
layer as well.
* add toolchain file
* add linux to preview targets
* add stub for dfu number in cmake
* excluded unity runner per default
* Added esp_attr.h and esp_partition.h linux stubs
* component.cmake check list for emptyness
* added switch for linux in unity cmake file
* Added Linux host example app
From now on, we have two tags for esp_flash tests:
- [esp_flash] for main flash chip only tests
- [esp_flash_3] for tests with external flash chips
To Run all tests, type `[esp_flash`; to run tests for main flash chip
only, type `[esp_flash].
There is a periodically yield in the esp_flash driver, to ensure the
cache will not be disabled for too long on ESP32.
On ESP32-S2 and later, we need to support more different kind of yield:
1. polling conditions, including timeout, SW read request, etc.
2. wait for events, including HW done/error/auto-suspend, timeout
semaphore, etc.
The check_yield() and yield() is separated into two parts, because we
may need to insert suspend, etc. between them.
1. The 2nd bootloader always call `rom_spiflash_unlock()`, but never help to clear the WEL bit when exit. This may cause system unstability.
This commit helps to clear WEL when flash configuration is done.
**RISK:** When the app starts, it didn't have to clear the WEL before it actually write/erase. But now the very first write/erase operation should be done after a WEL clear. Though the risk is little (all the following write/erase also need to clear the WEL), we still have to test this carefully, especially for those functions used by the OTA.
2. The `rom_spiflash_unlock()` function in the patch of ESP32 may (1) trigger the QPI, (2) clear the QE or (3) fail to unlock the ISSI chips.
Status register bitmap of ISSI chip and GD chip:
| SR | ISSI | GD25LQ32C |
| -- | ---- | --------- |
| 0 | WIP | WIP |
| 1 | WEL | WEL |
| 2 | BP0 | BP0 |
| 3 | BP1 | BP1 |
| 4 | BP2 | BP2 |
| 5 | BP3 | BP3 |
| 6 | QE | BP4 |
| 7 | SRWD | SRP0 |
| 8 | | SRP1 |
| 9 | | QE |
| 10 | | SUS2 |
| 11 | | LB1 |
| 12 | | LB2 |
| 13 | | LB3 |
| 14 | | CMP |
| 15 | | SUS1 |
QE bit of other chips are at the bit 9 of the status register (i.e. bit 1 of SR2), which should be read by RDSR2 command.
However, the RDSR2 (35H, Read Status 2) command for chip of other vendors happens to be the QIOEN (Enter QPI mode) command of ISSI chips. When the `rom_spiflash_unlock()` function trys to read SR2, it may trigger the QPI of ISSI chips.
Moreover, when `rom_spiflash_unlock()` try to clear the BP4 bit in the status register, QE (bit 6) of ISSI chip may be cleared by accident. Or if the ISSI chip doesn't accept WRSR command with argument of two bytes (since it only have status register of one byte), it may fail to clear the other protect bits (BP0~BP3) as expected.
This commit makes the `rom_spiflash_unlock()` check whether the vendor is issi. if so, `rom_spiflash_unlock()` only send RDSR to read the status register, send WRSR with only 1 byte argument, and also avoid clearing the QE bit (bit 6).
3. `rom_spiflash_unlock()` always send WRSR command to clear protection bits even when there is no protection bit active. And the execution of clearing status registers, which takes about 700us, will also happen even when there's no bits cleared.
This commit skips the clearing of status register if there is no protection bits active.
Also move the execute_flash_command to be a bootloader API; move
implementation of spi_flash_wrap_set to the bootloader
* partition api changed from spi_flash* API to
esp_partition* API and is abstracted as a C++
interface.
* The old nvs encryption is still possible
* changed default unit test app partition table
* Partitions coming from esp_partition API are
checked for generic flash encryption. If yes,
an error is returned since generic flash
encryption isn't compatible with nvs
encryption
* esp32, esp32s2 tests don't require nvs_flash
but mbedtls now
Closes IDF-1340
Closes IDF-858
* changing dependencies from unity->cmock
* added component.mk and Makefile.projbuild
* ignore test dir in gen_esp_err_to_name.py
* added some brief introduction of CMock in IDF
Sometimes the flash size read from bootloader is not correct. This may
forbid SPI Flash driver from reading the the area larger than the size
in bootloader header.
When the new config option is enabled, the latest configured
ESPTOOLPY_FLAHSIZE in the app header will be used to override the value
read from bootloader header.
Including:
1. Change the write bytes/read bytes parameter in the host driver into slicers to meet the requirements of complicated cases.
2. Refactor the esp_flash_api code a bit so that we can use the code in the ROM laster
3. Provide get_temp_buffer and release_temp_buffer in the os_functions when the buffer passed by application cannot be used directly.
4. Make timeout of operations configurable in the chip_driver.
5. Make dummy number configurable.
In commit 309376f51a, it seems like regression
was added to use ROM level API for disabling flash write protection. This
started random firmware crashes (on specific modules) with exception
`IllegalInstruction` during encrypted flash writes.
Fix here removes relevant ROM API call, since disabling flash write protection
is already ensured by caller of this API.
Closes https://github.com/espressif/esp-idf/issues/5467
Flash write operation is broken down into smaller chunk writes. Size
of this chunk was previously set to 8K but that in-turn meant cache and
non-IRAM resident interrupts could stay disabled upto ~24msec for 8K flash
write operation. If chunk size is brought down to 256 (typical flash page size)
then it brings down cache and non-IRAM interrupts disable duration to ~1msec.
Fix here keeps defaults same but provides configuration option to tweak the
setting based on application requirement.
The SPI bus lock on SPI1 introduces two side effects:
1. The device lock for the main flash requires the
`CONFIG_FREERTOS_SUPPORT_STATIC_ALLOCATION` to be selected, however this
option is disabled by default in earlier IDF versions. Some developers
may find their project cannot be built by their old sdkconfig files.
2. Usually we don't need the lock on the SPI1 bus, due to it's
restrictions. However the overhead still exists in this case, the IRAM
cost for static version of semaphore functions, and the time cost when
getting and releasing the lock.
This commit:
1. Add a CONFIG_SPI_FLASH_BYPASS_MAIN_LOCK option, which will forbid the
space cost, as well as the initialization of the main bus lock.
2. When the option is not selected, the bus lock is used, the
`CONFIG_FREERTOS_SUPPORT_STATIC_ALLOCATION` will be selected explicitly.
3. Revert default value of `CONFIG_FREERTOS_SUPPORT_STATIC_ALLOCATION`
to `n`.
introduced in 49a48644e4.
Closes https://github.com/espressif/esp-idf/issues/5046
We used to manually specify the CS id. However after the SPI bus lock is
introduced, the lock is responsible to assign the CS lines and provide
the CS id. The esp_flash driver now depends on the ID assigned by the
SPI bus lock, the configuration field is deprecated.
The issue is introduced in 571864e8ae. The
esp_flash API tries to clear the QE bit when the flash is not working in
quad modes.
However this introduces a regression, compared to earlier versions and
the legacy API. When the chip is not detected, the generic chip driver
is used, which cannot 100% handle the QE bit properly for all flash
vendors. There may be some flash chips (e.g. MXIC) that can be used in
dual modes by legacy API, but output wrong data when the esp_flash API
clears the QE bit in a wrong way.
This commit reverts the QE force clearing behavior, so that it's safer
for the generic chip driver to work under dual modes.
The default chip driver (chip_generic) use command 01H + 2 bytes to
clear the QE bit. However this will accidently change the configuration
register value of the MXIC chip.
MXIC chip driver is added to fix that.
Allocation of the temporary internal buffer will now repeat until a small enough buffer can be
allocated, and only fail if less than a 256 byte block of internal RAM is free.
Adds unit test for the same, and generic test utility for creating memory pressure.
Previously would try allocate buffer of minimum size 16KB not maximum size 16KB, causing
out of memory errors for any large reads, or if less than 16KB contiguous free heap.
Also, if using legacy API and internal allocation failed then implementation would abort()
instead of returning the error to the caller.
Added test for using large buffers in PSRAM.
Closes https://github.com/espressif/esp-idf/issues/4769
Also reported on forum: https://esp32.com/viewtopic.php?f=13&t=14304&p=55972
DISABLED_FOR_TARGETS macros are used
Partly revert "ci: disable unavailable tests for esp32s2beta"
This partly reverts commit 76a3a5fb48.
Partly revert "ci: disable UTs for esp32s2beta without runners"
This partly reverts commit eb158e9a22.
Partly revert "fix unit test and examples for s2beta"
This partly reverts commit 9baa7826be.
Partly revert "efuse: Add support for esp32s2beta"
This partly reverts commit db84ba868c.
1. add hal and low-level layer for timer group
2. add callback functions to handle interrupt
3. add timer deinit function
4. add timer spinlock take function
Tests for external flash chips used to controlled by macros, one bin for
one chip. And tests are done manually. This commit refactored the test
so that all 3 chips can all run in single test.
There used to be dummy phase before out phase in common command
transactions. This corrupts the data.
The code before never actually operate (clear) the QE bit, once it finds
the QE bit is set. It's hard to check whether the QE set/disable
functions work well.
This commit:
1. Cancel the dummy phase
2. Set and clear the QE bit according to chip settings, allowing tests
for QE bits. However for some chips (Winbond for example), it's not
forced to clear the QE bit if not able to.
3. Also refactor to allow chip_generic and other chips to share the same
code to read and write qe bit; let common command and read command share
configure_host_io_mode.
4. Rename read mode to io mode since maybe we will write data with quad
mode one day.
esp_partition_register_external did not call load_partitions, so if
it was called before any call to esp_partition_find, then the main
partition table would never be loaded. Introduce new function,
ensure_partitions_loaded, and call it both from esp_partition_find and
esp_partition_register_external.
Closes https://github.com/espressif/esp-idf/issues/4116
1. The translation should be first reviewed by technical reviewers;
2. and then by language reviewers.
For the translation for the first batch of files, please see !MR5620 and !MR5613
