Commit Graph

376 Commits

Author SHA1 Message Date
Adam Múdry
d42bde0378 feat(nvs): NVS tool add key consistency check 2023-10-09 15:58:48 +02:00
Adam Múdry
c276e2ca05 refactor: NVS tool move integrity check from logger to another file 2023-10-09 15:58:48 +02:00
Laukik Hase
6c9a343e1b
fix(docs/nvs_part_gen): Show the HMAC scheme-specific commands only for supported SOCs 2023-10-06 11:33:12 +05:30
radek.tandler
4f659b70e1 fix(nvs_flash): fixed failing nvs_page test 2023-09-26 15:05:20 +00:00
radek.tandler
e8992f500c fix(nvs_flash): nvs_set with same key and different data type
Function now always rewrites old value under same key
regardless existing data type. Users requiring old API behaviour
can enable it by kconfig option CONFIG_NVS_LEGACY_DUP_KEYS_COMPATIBILITY
2023-09-26 15:05:20 +00:00
radek.tandler
f693b78f93 feature(nvs_flash) : added support iteration over namespace by index - review changes 2023-08-29 16:05:02 +02:00
Ivan Grokhotkov
92ff04ebea nvs: use 'handle' argument name for consistency 2023-08-29 13:59:34 +02:00
Cody P Schafer
f2af1c5305 nvs: support iteration over namespace by index
Users of the nvs API are likely to have `nvs_handle_t` in all cases, but
not all of them carry around the partition name and namespace name (as
they aren't needed after creating an `nvs_handle_t`).

Allow this common case to use nvs iteration without them tracking
additional data by using the `nvs_handle_t` to locate the namespace
index and the partition name by introducing an alterate to
`nvs_entry_find` called `nvs_entry_find_in_handle`, which operates
similarly except that it is given a `nvs_handle_t` instead of partition
and namespace strings.

This is somewhat more limited than the `nvs_entry_find` API as one
cannot examine all keys in a given partition.
2023-08-29 13:59:34 +02:00
Martin Vychodil
e44d4260ad Merge branch 'feature/storage_host_test_nvs_flash2' into 'master'
nvs_flash: finished migration of host tests to the linux emulator of esp_partition

See merge request espressif/esp-idf!23926
2023-08-28 20:20:16 +08:00
Armando
7dbd3f6909 feat(ci): Enable p4 example, test_apps and unit tests CI build 2023-08-24 12:51:19 +08:00
Adam Múdry
42ac85a6a1 Merge branch 'contrib/github_pr_11926' into 'master'
Support NVS image generation from CMake (GitHub PR)

Closes IDFGH-10702 and IDFGH-10542

See merge request espressif/esp-idf!25102
2023-08-15 00:20:11 +08:00
radek.tandler
28f82d8de7 refactor(nvs_flash): Host test migrated to esp_partition emulation for linux 2023-08-09 14:14:21 +02:00
radek.tandler
3ffb8f8ffd nvs: nvs_get_stats fixed test, improved comment 2023-08-08 08:43:19 +02:00
radek.tandler
ac6c03fad1 change(nvs_flash): nvs_get_stats extended, API documentation improved 2023-08-08 08:43:19 +02:00
Vincent Hamp
d975a26666 feat: Add support for NVS image generation from CMake
Closes: https://github.com/espressif/esp-idf/pull/11926
Closes: https://github.com/espressif/esp-idf/pull/11785

Co-authored-by: Nebojša Cvetković <nebkat@gmail.com>
2023-07-31 18:26:06 +02:00
Wang Zi Yan
3e4152cdcf docs: Update CN for nvs_flash.rst and flash-encryption.rst 2023-07-14 04:05:53 +00:00
Adam Múdry
098cab35e4 NVS: nvs_tool.py entry.data None check fix 2023-06-21 11:23:22 +02:00
KonstantinKondrashov
e72061695e all: Removes unnecessary newline character in logs
Closes https://github.com/espressif/esp-idf/issues/11465
2023-06-09 03:31:21 +08:00
Laukik Hase
72f703ccd4
nvs_flash: Extended test-app and host tests for the HMAC-based NVS encr-keys protection scheme 2023-05-23 13:55:59 +05:30
Laukik Hase
a06118012e
docs: Update nvs_flash docs for the HMAC-based NVS encr-keys protection scheme
- Also updated the `nvs_partition_generator` and `mass_mfg` tools
  documentation
2023-05-23 13:55:57 +05:30
Laukik Hase
9ac87fcc8b
nvs_partition_gen/mass_mfg: Support for HMAC-based scheme for generating NVS encr-keys 2023-05-23 13:55:55 +05:30
Laukik Hase
c1bed366ba
nvs_flash: Add support for HMAC-based NVS encryption keys protection scheme
- This features allows the NVS encryption keys to be derived and protected using
  the HMAC peripheral. Since the encryption keys are derived at runtime, they
  are not stored anywhere in the flash and hence this feature does not require
  a separate `nvs_keys` partition.
2023-05-23 13:55:52 +05:30
Linda
65ee4992ce docs: update the algorithm and key name from AES-XTS to XTS-AES 2023-05-15 17:54:50 +08:00
Laukik Hase
4ac95a33fd
nvs_flash: Migrate UTs to component test-apps 2023-04-26 10:21:03 +05:30
Ivan Grokhotkov
2a28061eaa Merge branch 'contrib/github_pr_11117' into 'master'
test_nvs_host: fix asan reported bugs (GitHub PR)

Closes IDFGH-9860

See merge request espressif/esp-idf!23172
2023-04-20 22:37:42 +08:00
Ivan Grokhotkov
f64ac33554
nvs: fix string/blob comparison lengths in host tests 2023-04-18 08:02:38 +02:00
Cody P Schafer
cbe906e53c
nvs_host_test: replace sprintf with snprintf
With some libc impls (ie: Xcode 13.1), sprintf is marked as deprecated
and causes the build with `-Werror` to fail. Switch the `sprintf` usage
in test_nvs.cpp to use `snprintf` to allow builds to complete without
warning.
2023-04-03 16:33:57 -04:00
Cody P Schafer
630343e4b9
test_nvs_host: fix asan reported bugs
- Use of `sizeof(uint32_t *)` instead of `sizeof(uint32_t)` in
   arguments
 - Use of `memcmp` without checking that length agreed (add `memeq`)
2023-04-03 16:30:02 -04:00
Jakob Hasse
bed81f450d ci(nvs_flash): Increase timeout of nvs_page_test
* nvs_page_test runs into timeout in CI. According to log,
  test ran run around halfway, so increasing from 10 to 90 seconds.

* Also increased timeout of nvs_host_test, just to be sure
2023-02-13 12:35:52 +08:00
Martin Vychodil
a2b0068478 Merge branch 'bugfix/linux_build_errors' into 'master'
fix(linux): fix build errors without --skip-setting-flags

See merge request espressif/esp-idf!22063
2023-01-20 14:46:36 +08:00
Ivan Grokhotkov
cef0744299 Merge branch 'bugfix/linux_target_gen_partition_table' into 'master'
multiple fixes for linux target

Closes IDF-6641

See merge request espressif/esp-idf!21993
2023-01-18 11:57:08 +08:00
Jakob Hasse
9991862a10 fix(linux): fix build errors and warnings, remove ignore file 2023-01-18 09:24:28 +08:00
Fu Hanxi
b02687a6c4 test: migrate linux target host test jobs into pytest-embedded 2023-01-16 10:25:55 +08:00
Ivan Grokhotkov
e7540dbe0e
docs: remove the outdated "IDF monitor doesn't work on linux" notes 2023-01-13 08:14:56 +01:00
Ivan Grokhotkov
ec8f38c9da
build system: generate the partition table by default for linux target
This fixes the issue that "idf.py partition-table" had to be run
manually in order for the partition table to be generated, when
building for linux target.
2023-01-13 08:14:56 +01:00
Marius Vikhammer
7f0aa7b534 docs: updated path to nvs_tool.py in chinese doc. 2023-01-06 11:00:17 +08:00
Ivan Grokhotkov
61c5fc21e1
docs: fix nvs-related docs build issue 2023-01-05 09:28:57 +01:00
Adam Múdry
40d1eff2bd Merge branch 'feature/nvs_tool_refinement' into 'master'
nvs: Partition tool refinement

Closes IDF-6061

See merge request espressif/esp-idf!20683
2023-01-05 06:39:01 +08:00
mofeifei
0fc2fbe83f docs: update cn trans nvs_partition_tool/readme 2023-01-04 18:32:30 +01:00
Adam Múdry
87bdcc89d0 nvs: NVS tool refinement
add: JSON output
add: integrity check - correct number of free pages check
change: split to separate files
2023-01-04 12:44:22 +01:00
Jakob Hasse
e6ca481623 bugfix(nvs_flash): fixed potential memory leak in nvs::Storage::init() 2022-12-15 15:37:21 +08:00
Martin Vychodil
00f721f746 Merge branch 'refactor/nvs_allocatable_objects' into 'master'
refactor(nvs): custom allocator for all objects allocated in NVS

Closes IDF-6308

See merge request espressif/esp-idf!21239
2022-12-14 17:35:13 +08:00
Jakob Hasse
91f04aa877 refactor(nvs): custom allocator for all objects allocated in NVS 2022-12-05 18:59:33 +01:00
Jakob Hasse
ec986651db bugfix(nvs_flash): fixed wrong error check after open_nvs_handle
Closes https://github.com/espressif/esp-idf/issues/10240
2022-12-01 18:05:56 +01:00
radek.tandler
ca7f073e7b nvs_flash/host_test: migrated to the CMake build system and nvs flash Linux implementation
Host tests of nvs_flash eligible to run in Linux implementation of nvs flash were migrated. Remaining test cases
were left in original folder. Migrated test cases use CMake instead of make.
2022-11-16 14:15:23 +01:00
Martin Vychodil
c9c7573f71 Storage: Partition APIs moved to the new component 'esp_partition'
All the partition handling API functions and data-types were moved from the 'spi_flash' component to the new one named 'esp_partition'. See Storage 5.x migration guide for more details
2022-11-02 21:54:45 +01:00
Jakob Hasse
674fa9377f refactor(nvs): using define for namespace len instead of magic number
Closes https://github.com/espressif/esp-idf/issues/9671
2022-10-21 18:12:24 +02:00
Martin Vychodil
4799cc1439 Merge branch 'feature/nvs_partition_parse_tool' into 'master'
tools: NVS partition parser

Closes IDF-4049

See merge request espressif/esp-idf!18107
2022-09-21 22:16:13 +08:00
Mo Fei Fei
ec680a2a82 Docs: Add CN translation for nvs_partition_parser/README.rst 2022-09-20 14:02:23 +00:00
Matus Fabo
1c6edff560 add: data classes for parsing NVS binary data
add: functions to print formatted NVS data
add: color support via ANSI escape sequences
add: documentation
2022-09-20 14:02:23 +00:00
Ivan Grokhotkov
c392c06216
cmake: remove -D prefix from COMPILE_DEFINITIONS property
Unlike COMPILE_OPTIONS, COMPILE_DEFINITIONS CMake property assumes
values without the -D prefix, such as NAME or NAME=VAL.
Previously, IDF build system was passing COMPILE_DEFINITIONS build
property to CMake COMPILE_OPTIONS property, so -D prefix was not
a problem.
Now that COMPILE_DEFINITIONS CMake property is used, -D prefix has
to be removed.

(Note that this doesn't affect 'target_compile_definitions' function,
which strips -D prefix before adding the definition to the property.)
2022-09-20 11:08:02 +02:00
Jakob Hasse
1ab3e64cee doc (nvs): added note about maximum possible namespaces 2022-08-24 15:42:27 +02:00
Jakob Hasse
d6f3666875 Merge branch 'bugfix/nvs_flash_linux_target_cleanups' into 'master'
nvs_flash: cleanups related to linux target

See merge request espressif/esp-idf!19570
2022-08-19 21:07:48 +08:00
Ivan Grokhotkov
c6e9a0c350
esp_common: make esp_err_to_name and ESP_ERROR_CHECK usable on Linux 2022-08-18 18:16:45 +02:00
Ivan Grokhotkov
4e0fa26e64
nvs_flash: link the host test against libbsd when building on Linux
...and clean up the linking rule
2022-08-17 12:20:37 +02:00
Ivan Grokhotkov
7a1e19edf1
nvs_flash: replace strncpy + manual null termination with strlcpy
Since libbsd is now a build dependency on Linux, strncpy can be
replaced with the safer and less verbose strlcpy.
2022-08-17 12:20:36 +02:00
Ivan Grokhotkov
bee241b0e2
nvs_flash: remove leftover ESP_ERROR_CHECKS
Similar to the asserts, these are replaced by error checks.
2022-08-17 12:20:36 +02:00
Ivan Grokhotkov
02661081be
nvs_flash: make nvs_flash_init usable for Linux target
Since the partition API is now supported for Linux target builds,
this function can now be used.
2022-08-17 12:20:36 +02:00
Ivan Grokhotkov
bfb6f31872
nvs_flash: remove unnecessary check that the data pointer is in DROM
spi_flash driver knows how to write data from cache (DROM or PSRAM)
into flash, so the extra check in nvs_flash is unnecessary.
Besides, the hardcoded address limit (0x3ff00000) is wrong for some
of the newer chips.
2022-08-17 12:20:36 +02:00
Ivan Grokhotkov
08e41973f0
nvs_flash: update intrusive_list for compatibility with C++17
std::iterator is deprecated since C++17, the code produces a warning
when compiled with clang and libc++.
2022-08-17 12:20:36 +02:00
simon.chupin
35dda59209 tools: remove the dependency on the future package 2022-08-09 16:46:58 +02:00
Ivan Grokhotkov
401c10ecfb build system: re-add -Wno-format as private flag for some components 2022-08-03 16:42:47 +04:00
Shubham Kulkarni
3689035b7c tests: Fix CI build failures 2022-08-03 05:18:37 +00:00
Angus Gratton
bb141956df nvs_partition_gen: Allow up to 4000 byte strings with NVS V2
As documented, the limit for NVS format v2 is 4000 bytes for a string.
2022-07-22 15:43:19 +10:00
intern
4866e91398 docs: update nvs encryption cn trans 2022-07-18 14:40:56 +08:00
Fu Hanxi
c0568611dd docs: changes docs supported targets tables 2022-07-14 08:26:32 +08:00
Djordje Nedic
0331d0aa63 tools: Introduce support for blank lines in config and value files for mfg utility
Some users have requested this feature.

In order to avoid complete refactoring or introducing more code complexity, a design choice to create temporary files without blank lines is made.

Additionally, an extension check is added and there are multiple smaller code style and structure improvements.

Closes https://github.com/espressif/esp-idf/issues/8421
2022-07-08 13:22:35 +00:00
Cao Sen Miao
a690a87829 spi_flash: Remove legacy spi_flash drivers 2022-07-01 11:01:34 +08:00
Cao Sen Miao
64147c3794 CXX: Make spi_flash related part works 2022-07-01 10:59:34 +08:00
Ivan Grokhotkov
87180907f7 Merge branch 'bugfix/nvs_part_ptr_wrong_init' into 'master'
bugfix (nvs_flash): Fix nvs_flash_init_partition_ptr

Closes IDFGH-7503 and IDFGH-7152

See merge request espressif/esp-idf!18503
2022-06-23 21:23:39 +08:00
Djordje Nedic
bbc3add0e3 tools: overhaul csv handling for mfg_gen and nvs_partition_gen
This fixes the issue where multiline strings and strings with delimiters inside the nvs input csv file were incorrectly parsed, and adds back the ability to add comment lines anywhere in the CSV file.

The issue stems from the move away from the python built in csv module to manual parsing, which was made after moving away from using the csv module to parse mfg data.

This reverts back to using the csv module for parsing and writing csv data in both mfg_gen and nvs_partition_gen, fixes the original issue in mfg_gen and improves code quality which makes the code more readable and maintainable.

Closes https://github.com/espressif/esp-idf/issues/7175
2022-06-15 12:31:49 +00:00
ioio
a759e00958 bugfix (nvs_flash): Fix nvs_flash_init_partition_ptr
Added unit test case.

Closes https://github.com/espressif/esp-idf/pull/8755

Signed-off-by: Jakob Hasse <jakob.hasse@espressif.com>
2022-06-15 10:38:56 +08:00
Djordje Nedic
facab8c5a7 tools: Increase the minimal supported CMake version to 3.16
This updates the minimal supported version of CMake to 3.16, which in turn enables us to use more CMake features and have a cleaner build system.
This is the version that provides most new features and also the one we use in our latest docker image for CI.
2022-06-01 06:35:02 +00:00
Anton Maklakov
25cee26784 Merge branch 'bugfix/storage-warnings' into 'master'
nvs: minor warning fixes

See merge request espressif/esp-idf!18313
2022-05-31 13:46:13 +08:00
Jakob Hasse
97c149f73d Merge branch 'refactor/nvs_iterator_api' into 'master'
NVS: New interface for iterator functions

Closes IDFGH-6149

See merge request espressif/esp-idf!18081
2022-05-31 07:47:26 +08:00
Ivan Grokhotkov
87ec752bda Merge branch 'docs/fix_nvs_encryption_docs' into 'master'
doc (nvs_flash): improved NVS partition flash documentation

Closes IDFGH-6549

See merge request espressif/esp-idf!18004
2022-05-30 16:13:12 +08:00
Anton Maklakov
b3a6c3bcb5 nvs: suppress -Wstringop-truncation warning 2022-05-30 13:11:25 +07:00
Jakob Hasse
ad184e979a refactor (nvs)!: New interface for iterator functions
Closes https://github.com/espressif/esp-idf/issues/7826

* nvs_entry_find(), nvs_entry_next() and nvs_entry_info()
  return error codes now
* nvs_entry_find() and nvs_entry_next() access/modify iterator via
  parameters, instead of returning an new iterator.

Added appropriate documentation in Chinese and English
2022-05-30 09:28:42 +08:00
Matus Fabo
58cca07104 add: Kconfig assert or errorcode option
add: private include header
add: macro encapsulation for assertion or error check
add: ESP_FAIL return code documentation in public headers
change: replaced all assertions by NVS_ASSERT_OR_RETURN macro
change: few internal function return values from void to esp_err_t
change: ESP_ERR_NVS_VALUE_TOO_LONG macro comment
2022-05-28 20:29:05 +02:00
Jakob Hasse
dd3c5c3cbb doc (nvs_flash): improved NVS partition flash documentation
Closes https://github.com/espressif/esp-idf/issues/8196
2022-05-16 14:13:43 +08:00
Jakob Hasse
e899edd793 Merge branch 'bugfix/nvs_coverity_issues' into 'master'
bugfix (nvs): Fixed issues found by Coverity

Closes IDF-4391

See merge request espressif/esp-idf!16951
2022-01-29 21:52:18 +00:00
Jakob Hasse
4c8c2a8079 bugfix (nvs): Fixed issues found by Coverity
*  Fixed potential memory leak
*  Fixed wrong strncpy usage
*  Fixed potential out of bounds access
2022-01-27 19:30:58 +08:00
Jakob Hasse
1d4061d3f2 refactor (nvs_flash)!: removed mbedtls as default dependency
mbedtls is only required in NVS if NVS encryption is enabled.
Hence the mbedtls dependency is now only added if encryption
is enabled, otherwise it is not added.

BREAKING CHANGE: Removed mbedtls as default dependency
2022-01-26 17:32:23 +08:00
Michael (XIAO Xufeng)
43643716a2 bugfix (nvs): fixed wrong pointer check 2021-12-13 10:42:45 +08:00
morris
2e008825a9 Merge branch 'feat/rm_makefiles' into 'master'
Build & config: Remove leftover files from the unsupported make build system

Closes IDF-4273 and IDF-2827

See merge request espressif/esp-idf!15829
2021-11-16 03:44:41 +00:00
Roland Dobai
766aa57084 Build & config: Remove leftover files from the unsupported "make" build system 2021-11-11 15:32:36 +01:00
Mahavir Jain
e67128dd0a nvs: add config to ignore "encrypted" flag of nvs partitions
This is to allow having pre IDF v4.3 behavior where "encrypted"
flag was not being checked for NVS partitions.

It is recommended to enable this new config only if you have
production devices where NVS partition was being set with "encrypted"
flag by mistake.

Please see commit aca9ec28b3 which
introduced check to not allow NVS partitions with "encrypted" flag set.

More discussion on this at:
https://github.com/espressif/esp-idf/issues/5747#issuecomment-956223024
https://github.com/espressif/esp-idf/issues/7839#issuecomment-961477667

Closes https://github.com/espressif/esp-idf/issues/7839
Closes IDFGH-6162
2021-11-11 15:20:57 +05:30
Shivani Tipnis
d8b1c3d44a nvs_tool: Update Readme with supports of multiline strings and comments in the CSV file
Closes https://github.com/espressif/esp-idf/issues/7175
2021-09-16 11:11:19 +05:30
Jakob Hasse
46e59e7194 [mocks]: moved already added spi_flash mocks
[nvs]: page test works with new overriden mock

[spi_flash]: Removed mocking code from CMake

* Renamed nvs test to be consistent with other
  host test
2021-08-18 11:05:51 +08:00
Jakob Hasse
7029f0537e [build system]: Fixed common requirement for Linux 2021-08-13 16:50:29 +08:00
Jakob Hasse
84d184bdb0 Merge branch 'bugfix/nvs_corrupted_storage_crashes_app' into 'master'
[NVS]: fix crashes from entry state 1

See merge request espressif/esp-idf!14548
2021-08-06 09:14:21 +00:00
Jakob Hasse
0b86f81847 [nvs]: use real esp_common on host tests 2021-08-03 19:47:17 +08:00
Jakob Hasse
4dd88329c1 [esp_rom]: Partially buildable for linux
The following files have been ported:
* esp_rom_crc.h
* esp_rom_sys.h
* esp_rom_efuse.h (mostly no-ops)
* esp_rom_md5.h

Integrated Linux-based rom implementation into log
and NVS component.

Added brief host tests for ROM to ensure basic
consistency on Linux.

Added ROM printf host unit tests.

Temporarily added reset reason for Linux in ROM.
2021-08-03 12:03:24 +08:00
Jakob Hasse
de6b54de66 [nvs_flash]: Entries with state == 1 don't crash 2021-07-30 18:33:27 +08:00
morris
4c0ddd34b4 ci: added script to check api violations 2021-06-22 11:28:01 +08:00
Anton Maklakov
e5faf4f924 ci: fix test_build_system* fails 2021-06-07 18:48:38 +07:00
Jakob Hasse
68393c41c4 doc: linux host test documentation 2021-04-23 09:27:35 +08:00
Jakob Hasse
2fb5b42eb1 [nvs] add nvs page host test to CI
Closes IDF-2425
2021-04-20 14:40:16 +08:00
Jakob Hasse
d52a21de5c Random NVS fixes
* Checking Lock::init() now
* fixed typos in nvs_flash.h
* Added missing parameter checks in nvs encryption
  function

* Closes IDF-1462
* Closes IDF-2900
2021-03-08 14:41:53 +08:00
Angus Gratton
d6f4d99d93 core system: Fix warnings in compilation when assertions are disabled
Adds a CI config for hello world that sets this, to catch future regressions
2021-03-03 10:26:57 +11:00