Some users have requested this feature.
In order to avoid complete refactoring or introducing more code complexity, a design choice to create temporary files without blank lines is made.
Additionally, an extension check is added and there are multiple smaller code style and structure improvements.
Closes https://github.com/espressif/esp-idf/issues/8421
This fixes the issue where multiline strings and strings with delimiters inside the nvs input csv file were incorrectly parsed, and adds back the ability to add comment lines anywhere in the CSV file.
The issue stems from the move away from the python built in csv module to manual parsing, which was made after moving away from using the csv module to parse mfg data.
This reverts back to using the csv module for parsing and writing csv data in both mfg_gen and nvs_partition_gen, fixes the original issue in mfg_gen and improves code quality which makes the code more readable and maintainable.
Closes https://github.com/espressif/esp-idf/issues/7175
* When deinitializing or erasing a partition,
nvs used to close all handles instead of only
the current partition's handles.
This is fixed now
* Added a unit test for that case
Closes FCS-533
* partition api changed from spi_flash* API to
esp_partition* API and is abstracted as a C++
interface.
* The old nvs encryption is still possible
* changed default unit test app partition table
* Partitions coming from esp_partition API are
checked for generic flash encryption. If yes,
an error is returned since generic flash
encryption isn't compatible with nvs
encryption
* esp32, esp32s2 tests don't require nvs_flash
but mbedtls now
Closes IDF-1340
Closes IDF-858
This prevents wear and tear on the flash, and it also is faster in some
cases since the read-out of flash is a cheaper operation than the erasure
of flash. Some library modules (such as the esp_wifi) write out to NVS
upon every initialization without checking first that the existing value
is the same, and this speeds up initialization of modules that make
these design choices and moves it into a centralized place.
The comparison functions are based on the read-out functions of the same
name, and changes out the memcpy(...) operations for memcmp(...)
operations.
Signed-off-by: Tim Nordell <tim.nordell@nimbelink.com>
Earlier eraseItem function in Storage class would do lazy cleanup of
multi-page blobs if called using type "ANY" instead of "BLOB". It used to
just delete BLOB data and index would remain as is. Any subsequent read
would delete index entry as well. This however would return a valid
length without error if nvs_get_blob API was just used for finding
length and not reading the complete blob. This change fixes this issue.
Closes https://github.com/espressif/esp-idf/issues/3255
Previously when HashList was removing items, HashListBlocks were
removed lazily. This resulted in empty HashListBlocks dangling around
in full pages, even when all items have been erased. These blocks
would only be deleted when NVS was re-initialized
(nvs_flash_deinit/nvs_flash_init).
This change does eager cleanup instead, based on the code from
@negativekelvin offered in
https://github.com/espressif/esp-idf/issues/1642#issuecomment-367227994.
Closes https://github.com/espressif/esp-idf/issues/1642.
Marking a page full does not skip it from page selection process and the
same page might get returned if there is no other page with more unused
entries. Added a check for the same while storing blobs.
Fixes: https://github.com/espressif/esp-idf/issues/2313
This change adds a check for compatibility between the nvs version
found on nvs flash and the one assumed by running code during nvs
initialization. Any mismatch is reported to the user using new error
code ESP_ERR_NVS_NEW_VERSION_FOUND.
This change removes the earlier limitation of 1984 bytes for storing data-blobs.
Blobs larger than the sector size are split and stored on multiple sectors.
For this purpose, two new datatypes (multi-page index and multi-page data) are
added for entries stored in the sectors. The underlying read, write, erase and find
operations are modified to support these large blobs. The change is transparent
to users of the library and no special APIs need to be used to store these large
blobs.
Currently when page is being freed, items are individually moved from
FREEING page to ACTIVE page and erased. If power-off happens during the
process, the remaining entries are moved to ACTIVE page during recovery.
The problem with this approach is there may not be enough space on
ACTIVE page for all items if an item was partially written before
power-off and erased during recovery. This change moves all the items
from FREEING to ACTIVE page and then erased the FREEING page, If
power-off happens during the process, then ACTIVE page is erased and the
process is restarted.
Ideally suited for generating a binary externally, containing key-value pairs specific
to device manufacturers. Utility allows creation of a binary, compatible
with NVS structure, which can be separately flashed onto a new
partition. This helps device manufacturers set different values for
different devices, e.g. serial numbers, but using a single firmaware
image.
Current code for recovery after power-off do not clean-up partially
erased items for FULL pages. If the erasure was part of modification
operation, this gets luckily cleaned-up because of duplicate detection
logic. For erase-only operation, the problem still exists. This patch
adds the recovery for FULL pages also.
Closes TW<20284>
When erasing a variable length item with an incorrect CRC32, the span
value of the item can not be trusted, so the item will be erased with
span = 1. Subsequent entries represent the data of the variable
length item, and these will be treated as separate items. For each
entry CRC32 is checked, the check most likely fails (because the
entry contains arbitrary data, and not a proper NVS item), and the
entry is erased. Erase function assumed that every item should be
present in cache, but it is not the case for the entries which are
just parts of item’s payload. This change allows for the item to be
not found in the hashlist, if the CRC32 check fails.
Current page selection algorithm selects a page for compaction based on just erased counts
and gives up when it does not find any page with erased count greater than 0. This is
problematic since the current allocation procedure skips the active page if there is not
enough room for the item in that page leaving free chunks on the pages. This change modifies
the algorithm to consider both erased as well as free counts on the candidate pages.
Closes TW<20297>
Users needs functions to count the number of free and used entries.
1. `nvs_get_stats()` This function return structure of statistic about the uspace NVS.
(Struct: used_entries, free_entries, total_entries and namespace_count)
2. `nvs_get_used_entry_count()` The second function return amount of entries in the namespace (by handler)
3. Added unit tests.
Closes TW<12282>
Previously NVS did check CRC values of key-value pairs on the active
page, but the check for full pages was missing. This adds the necessary
check and a test for it.
