This patch modifies the OTA implementation to incremental erase flash sectors
as they are written, rather than bulk erasing them all at once when OTA begins.
This solves several problems:
- The bulk flash erase can take so long that it triggers the watchdog timer.
Disabling the watchdog is an undesirable risk.
- The bulk erase interferes with the responsiveness of the system to user input.
- Incremental erasing eliminates the need to pass the firmware image size to
esp_ota_begin to avoid erasing more flash sectors than required.
- When installing an OTA received via HTTP, the size of the firmware image is
known when the content-length header is received. Calling esp_ota_begin at that
time causes a long delay to do the bulk erase which causes the TCP transfer
speed to be measurably slowed when the transfer resumes after the erase.
Incremental erasing eliminates this TCP slowdown.
Update:
Rework so erase optimization is only applied when image_size is set to
OTA_WITH_SEQUENTIAL_WRITES #5246
Merges https://github.com/espressif/esp-idf/pull/5246
* changing dependencies from unity->cmock
* added component.mk and Makefile.projbuild
* ignore test dir in gen_esp_err_to_name.py
* added some brief introduction of CMock in IDF
Commit fc03161f updated esp_ota_get_app_elf_sha256 to store and return
X number of bytes of the hash, but the test case still expected 64 bytes.
Updated test case to use CONFIG value for expected length.
DISABLED_FOR_TARGETS macros are used
Partly revert "ci: disable unavailable tests for esp32s2beta"
This partly reverts commit 76a3a5fb48.
Partly revert "ci: disable UTs for esp32s2beta without runners"
This partly reverts commit eb158e9a22.
Partly revert "fix unit test and examples for s2beta"
This partly reverts commit 9baa7826be.
Partly revert "efuse: Add support for esp32s2beta"
This partly reverts commit db84ba868c.
This MR improves existing flash encryption document to provide simplified steps
Adds two new modes for user: Development & Release
Adds a simple example
Supports encrypted write through make command
Do not include bootloader in flash target when secure boot is enabled.
Emit signing warning on all cases where signed apps are enabled (secure
boot and signed images)
Follow convention of capital letters for SECURE_BOOT_SIGNING_KEY
variable, since it is
relevant to other components, not just bootloader.
Pass signing key and verification key via config, not requiring
bootloader to know parent app dir.
Misc. variables name corrections
!4452 used setting LINK_LIBRARIES and INTERFACE_LINK_LIBRARIES to link
components built under ESP-IDF build system. However, LINK_LIBRARIES does
not produce behavior same as linking PRIVATE. This MR uses the new
signature for target_link_libraries directly instead. This also moves
setting dependencies during component registration rather than after all
components have been processed.
The consequence is that internally, components have to use the new
signature form as well. This does not affect linking the components to
external targets, such as with idf_as_lib example. This only affects
linking additional libraries to ESP-IDF libraries outside component processing (after
idf_build_process), which is not even possible for CMake<v3.13 as
target_link_libraries is not valid for targets not created in current
directory. See https://cmake.org/cmake/help/v3.13/policy/CMP0079.html#policy:CMP0079
Using xxx_periph.h in whole IDF instead of xxx_reg.h, xxx_struct.h, xxx_channel.h ... .
Cleaned up header files from unnecessary headers (releated to soc/... headers).
This MR removes the common dependency from every IDF components to the SOC component.
Currently, in the ``idf_functions.cmake`` script, we include the header path of SOC component by default for all components.
But for better code organization (or maybe also benifits to the compiling speed), we may remove the dependency to SOC components for most components except the driver and kernel related components.
In CMAKE, we have two kinds of header visibilities (set by include path visibility):
(Assume component A --(depends on)--> B, B is the current component)
1. public (``COMPONENT_ADD_INCLUDEDIRS``): means this path is visible to other depending components (A) (visible to A and B)
2. private (``COMPONENT_PRIV_INCLUDEDIRS``): means this path is only visible to source files inside the component (visible to B only)
and we have two kinds of depending ways:
(Assume component A --(depends on)--> B --(depends on)--> C, B is the current component)
1. public (```COMPONENT_REQUIRES```): means B can access to public include path of C. All other components rely on you (A) will also be available for the public headers. (visible to A, B)
2. private (``COMPONENT_PRIV_REQUIRES``): means B can access to public include path of C, but don't propagate this relation to other components (A). (visible to B)
1. remove the common requirement in ``idf_functions.cmake``, this makes the SOC components invisible to all other components by default.
2. if a component (for example, DRIVER) really needs the dependency to SOC, add a private dependency to SOC for it.
3. some other components that don't really depends on the SOC may still meet some errors saying "can't find header soc/...", this is because it's depended component (DRIVER) incorrectly include the header of SOC in its public headers. Moving all this kind of #include into source files, or private headers
4. Fix the include requirements for some file which miss sufficient #include directives. (Previously they include some headers by the long long long header include link)
This is a breaking change. Previous code may depends on the long include chain.
You may need to include the following headers for some files after this commit:
- soc/soc.h
- soc/soc_memory_layout.h
- driver/gpio.h
- esp_sleep.h
The major broken include chain includes:
1. esp_system.h no longer includes esp_sleep.h. The latter includes driver/gpio.h and driver/touch_pad.h.
2. ets_sys.h no longer includes soc/soc.h
3. freertos/portmacro.h no longer includes soc/soc_memory_layout.h
some peripheral headers no longer includes their hw related headers, e.g. rom/gpio.h no longer includes soc/gpio_pins.h and soc/gpio_reg.h
BREAKING CHANGE
1. separate rom include files and linkscript to esp_rom
2. modefiy "include rom/xxx.h" to "include esp32/rom/xxx.h"
3. Forward compatible
4. update mqtt
Added:
* set a secure version in app/bootloader.
* description anti-rollback to ota part
* emulate the secure_version write and read operations
* efuse_em partition.
* a description about a rollback for native_ota_example.
Closes: TW26335
On enabling flash encryption, OTA fails with magic byte error in the second chunk
if the http data is split into two or more chunks and first chunk is less than 16.
Added a new structure esp_app_desc_t. It has info about firmware:
version, secure_version, project_name, time/date build and IDF version.
Added the ability to add a custom structure with a description of the firmware.
The esp_app_desc_t is located in fixed place in start of ROM secotor. It is located after structures esp_image_header_t and esp_image_segment_header_t.
app_version is filed from PROJECT_VER variable (if set in custom make file) or PROJECT_PATH/version.txt or git repo (git describe).
Add API to get app_desc from partition.
New unity component can be used for testing other applications.
Upstream version of Unity is included as a submodule.
Utilities specific to ESP-IDF unit tests (partitions, leak checking
setup/teardown functions, etc) are kept only in unit-test-app.
Kconfig options are added to allow disabling certain Unity features.
Routine `spi_flash_cache2phys` sometimes return incorrect value,
resulting in failure in getting currently running ota partition.
This in turn aborts firmware update process.This issue was more
prominent with SPIRAM enabled cases.
Fix ensures proper cache guards during `spi_flash_cache2phys`, and
also for few other similar APIs.
In addition, `esp_ota_get_running_partition` has also been optimized
to save currently running partition for subsequent invocations.
Fixes https://github.com/espressif/esp-idf/issues/2451
Allows OTA updates to be secured via signature checks, without requiring the overhead or complexity
of a full secure boot implementation.
Uses same signing mechanisms (build system and/or espsecure.py as Secure Boot).
Requires:
* [ ] More testing
* [ ] Documentation
Added bootloader_common_get_sha256_of_partition() and esp_partition_get_sha256() - get or calculate SHA-256
digest for app and data partitions.
Added bootloader_sha256_hex_to_str() - helps to print SHA-256 digest
Added esp_partition_check_identity() - compares two partitions by SHA-256 digest
Refactoring a function esp_image_load() in bootloader space to esp_image_verify() and
bootloader_load_image(). Old name function esp_image_load is deprecated
and will remove in V4.0 version.
spi_flash/sim: Fix error test_host. Add stub for bootloader_common_get_sha256_of_partition in sim/stubs
Add a command `make erase_ota` - erases otadata if it there is in the
partition table otherwise error 'Partition table does not have ota_data partition'.
Closes https://github.com/espressif/esp-idf/issues/1821
in the include file esp_ota_ops.h "esp_partition.h" is included.
This is from spi_flash so component that requires app_update also will need that
this fixes this error:
In file included from ../components/esp32-homie/ota.c:11:0:
/esp-idf/components/app_update/include/esp_ota_ops.h:22:27: fatal error: esp_partition.h: No such file or directory
Signed-off-by: Nicola Lunghi <25422924+nicola-lunghi@users.noreply.github.com>
Merges https://github.com/espressif/esp-idf/pull/1947
* Philosophical: "explicit is better than implicit".
* Practical: Allows useful errors if invalid directories given in components as the defaults aren't
always used. Also trims the -I path from a number of components that have no actual include
directory.
* Simplifies knowing which components will be header-only and which won't
Makes app image booting more reliable (256-bit rather than 8-bit verification.)
Some measurements, time to boot a 655KB app.bin file and run to app_main() execution.
(All for rev 1 silicon, ie no 340ms spurious WDT delay.)
80MHz QIO mode:
before = 300ms
after = 140ms
40MHz DIO mode:
before = 712ms
after = 577ms
40MHz DIO mode, secure boot enabled
before = 1380ms
after = 934ms
(Secure boot involves two ECC signature verifications (partition table, app) that take approx 300ms each with 80MHz CPU.)
Even if firmware is compiled without CONFIG_FLASH_ENCRYPTION_ENABLED
Rayionale: CONFIG_FLASH_ENCRYPTION_ENABLED controls whether boot loader
generates keys for encryption or not, but flash encryption can be
configured externally. With this change, it's possible to have boot
loader not generate keys but still have encryption working.
Also fix use of it->part
Verify 0xE9 magic byte on first write, verify entire image before
switching.
Enable verification for secure boot signature (was using invalid ifdef
guard)
bugfix/init_factory_bin: fix inconsistency between bootloader and OTA code
init_factory_bin: 2nd bootloader use a wrong logic select factory image to boot,and some bug in OTA code
See merge request !266
