handling (only when nvs encryption is enabled)
* NVS Encryption will now be turned on by default with flash encryption
* Updated the flash encryption example to shocase NVS encryption
along with information on how to configure and use NVS encryption
* Updated respective test case
* Added two partition tables for NVS encryption
i) Table 1- Single factory app, no OTA, encrypted NVS
ii) Table 2- Factory app, Two OTA, encrypted NVS
* General tests like page loading from flash
* Rough test of fixed-size data types
* Rough test of blob read
* Added coverage target in cmake, also accessible
via `idf.py coverage`
* Fixed unsigned comparison in comp. enum table
* introducing temporary LINUX_TARGET define
* When deinitializing or erasing a partition,
nvs used to close all handles instead of only
the current partition's handles.
This is fixed now
* Added a unit test for that case
Closes FCS-533
* partition api changed from spi_flash* API to
esp_partition* API and is abstracted as a C++
interface.
* The old nvs encryption is still possible
* changed default unit test app partition table
* Partitions coming from esp_partition API are
checked for generic flash encryption. If yes,
an error is returned since generic flash
encryption isn't compatible with nvs
encryption
* esp32, esp32s2 tests don't require nvs_flash
but mbedtls now
Closes IDF-1340
Closes IDF-858
* changing dependencies from unity->cmock
* added component.mk and Makefile.projbuild
* ignore test dir in gen_esp_err_to_name.py
* added some brief introduction of CMock in IDF
* closes IDF-1135:
same namespace was used in different tests which
could lead to conflicts when re-running tests
* removes duplicated functions declarations
* correct argument order for test case macros
1. The translation should be first reviewed by technical reviewers;
2. and then by language reviewers.
For the translation for the first batch of files, please see !MR5620 and !MR5613
Rename and add multiple kconfig compiler options. New compiler options
COMPILER_OPTIMIZATION_PERF and COMPILER_OPTIMIZATION_NONE have been added.
Optimize "Debug" and "Release" options to "Default" and "Size" respectively.
This commit also does the following:
- The COMPILER_OPTIMIZATION_PERF option introduced multiple bug.
This commit fixes those bugs.
- build.yml also updated to test for the new optimization options.
This prevents wear and tear on the flash, and it also is faster in some
cases since the read-out of flash is a cheaper operation than the erasure
of flash. Some library modules (such as the esp_wifi) write out to NVS
upon every initialization without checking first that the existing value
is the same, and this speeds up initialization of modules that make
these design choices and moves it into a centralized place.
The comparison functions are based on the read-out functions of the same
name, and changes out the memcpy(...) operations for memcmp(...)
operations.
Signed-off-by: Tim Nordell <tim.nordell@nimbelink.com>
Earlier eraseItem function in Storage class would do lazy cleanup of
multi-page blobs if called using type "ANY" instead of "BLOB". It used to
just delete BLOB data and index would remain as is. Any subsequent read
would delete index entry as well. This however would return a valid
length without error if nvs_get_blob API was just used for finding
length and not reading the complete blob. This change fixes this issue.
Closes https://github.com/espressif/esp-idf/issues/3255
Merges https://github.com/espressif/esp-idf/pull/3216
This should read "except for the new key-value pair" instead of
"expect for the new key-value pair".
Signed-off-by: Tim Nordell <tim.nordell@nimbelink.com>
1. separate rom include files and linkscript to esp_rom
2. modefiy "include rom/xxx.h" to "include esp32/rom/xxx.h"
3. Forward compatible
4. update mqtt
Previously when HashList was removing items, HashListBlocks were
removed lazily. This resulted in empty HashListBlocks dangling around
in full pages, even when all items have been erased. These blocks
would only be deleted when NVS was re-initialized
(nvs_flash_deinit/nvs_flash_init).
This change does eager cleanup instead, based on the code from
@negativekelvin offered in
https://github.com/espressif/esp-idf/issues/1642#issuecomment-367227994.
Closes https://github.com/espressif/esp-idf/issues/1642.
Currently, only erase operation performed by the application leads
to detection of NVS key partition as uninitialised. This change
adds additional checks for detecting partition as uninitialised,
when device boots first time right after encryption by bootloader.
When std::bind is used, it requires inclusion of <functional> header.
This was not mandatory with earlier versions of g++ (4.x), may be because they
had experimental support for c++11.
New unity component can be used for testing other applications.
Upstream version of Unity is included as a submodule.
Utilities specific to ESP-IDF unit tests (partitions, leak checking
setup/teardown functions, etc) are kept only in unit-test-app.
Kconfig options are added to allow disabling certain Unity features.
Marking a page full does not skip it from page selection process and the
same page might get returned if there is no other page with more unused
entries. Added a check for the same while storing blobs.
Fixes: https://github.com/espressif/esp-idf/issues/2313
This change adds a check for compatibility between the nvs version
found on nvs flash and the one assumed by running code during nvs
initialization. Any mismatch is reported to the user using new error
code ESP_ERR_NVS_NEW_VERSION_FOUND.
This change removes the earlier limitation of 1984 bytes for storing data-blobs.
Blobs larger than the sector size are split and stored on multiple sectors.
For this purpose, two new datatypes (multi-page index and multi-page data) are
added for entries stored in the sectors. The underlying read, write, erase and find
operations are modified to support these large blobs. The change is transparent
to users of the library and no special APIs need to be used to store these large
blobs.
Currently when page is being freed, items are individually moved from
FREEING page to ACTIVE page and erased. If power-off happens during the
process, the remaining entries are moved to ACTIVE page during recovery.
The problem with this approach is there may not be enough space on
ACTIVE page for all items if an item was partially written before
power-off and erased during recovery. This change moves all the items
from FREEING to ACTIVE page and then erased the FREEING page, If
power-off happens during the process, then ACTIVE page is erased and the
process is restarted.
* Philosophical: "explicit is better than implicit".
* Practical: Allows useful errors if invalid directories given in components as the defaults aren't
always used. Also trims the -I path from a number of components that have no actual include
directory.
* Simplifies knowing which components will be header-only and which won't
Ideally suited for generating a binary externally, containing key-value pairs specific
to device manufacturers. Utility allows creation of a binary, compatible
with NVS structure, which can be separately flashed onto a new
partition. This helps device manufacturers set different values for
different devices, e.g. serial numbers, but using a single firmaware
image.
Current code for recovery after power-off do not clean-up partially
erased items for FULL pages. If the erasure was part of modification
operation, this gets luckily cleaned-up because of duplicate detection
logic. For erase-only operation, the problem still exists. This patch
adds the recovery for FULL pages also.
Closes TW<20284>
When erasing a variable length item with an incorrect CRC32, the span
value of the item can not be trusted, so the item will be erased with
span = 1. Subsequent entries represent the data of the variable
length item, and these will be treated as separate items. For each
entry CRC32 is checked, the check most likely fails (because the
entry contains arbitrary data, and not a proper NVS item), and the
entry is erased. Erase function assumed that every item should be
present in cache, but it is not the case for the entries which are
just parts of item’s payload. This change allows for the item to be
not found in the hashlist, if the CRC32 check fails.
Current page selection algorithm selects a page for compaction based on just erased counts
and gives up when it does not find any page with erased count greater than 0. This is
problematic since the current allocation procedure skips the active page if there is not
enough room for the item in that page leaving free chunks on the pages. This change modifies
the algorithm to consider both erased as well as free counts on the candidate pages.
Closes TW<20297>
Users needs functions to count the number of free and used entries.
1. `nvs_get_stats()` This function return structure of statistic about the uspace NVS.
(Struct: used_entries, free_entries, total_entries and namespace_count)
2. `nvs_get_used_entry_count()` The second function return amount of entries in the namespace (by handler)
3. Added unit tests.
Closes TW<12282>
* Change snprintf for strlcat does not complain w/gcc7.2.0 and it is safer, thanks @projectgus
* Use proper quotes for character literals
Merges https://github.com/espressif/esp-idf/pull/1163
