Commit Graph

416 Commits

Author SHA1 Message Date
Guillaume Souchere
787a4ad6c9 Merge branch 'feat/use-singly-linked-hashmap' into 'master'
feat(heap): update hash map to use singly linked list

Closes IDFGH-9846

See merge request espressif/esp-idf!26441
2023-10-23 13:12:37 +08:00
Guillaume Souchere
07165308f6 feat(heap): update hash map to use singly linked list
Previously, the hash map was a doubly linked list but was never
using the characteristics of it.

This commit switches the hash map to a singly linked list instead

This commit also fixes memory leak in heap trace tests by setting
hashmap size to 10 for the tests (instead of the default value of 250)

See https://github.com/espressif/esp-idf/issues/11173
2023-10-19 14:50:05 +02:00
Guillaume Souchere
63952fe2f6 Merge branch 'feat/make-task-tracking-independant-from-poisoning' into 'master'
feat(heap): Dissociate heap poisoning from task tracking functionalities

Closes IDF-8332

See merge request espressif/esp-idf!26311
2023-10-18 20:18:22 +08:00
Guillaume Souchere
7492c862af feat(heap): Dissociate heap poisoning from task tracking
In order to enable CONFIG_HEAP_TASK_TRACKING, some kind
of poisoning had to be enabled (!HEAP_POISONING_DISABLED).
However since those functionalities don't seem to be related
in any way, this commit decouple them by removing
MULTI_HEAP_BLOCK_OWNER from poison_head_t in multi_heap_poisoning.c
and handling the block ownership in heap_caps.c instead.

Note that handling task tracking in multi_heap.c would necessitate
updating the ROM implementation of multi_heap.c as well. For this
reason, the task tracking feature has to be handled in heap_caps.c.
2023-10-18 06:43:22 +02:00
Guillaume Souchere
84c5d3e52f fix(heap): Fixed integrity check on used blocks by the tlsf component
This commit updates the tlsf submodule to include the modification made in the component
aiming to perform integrity check on all blocks (not only the free ones).
Added test to test the fix in test_apps/heap_tests.

Fixes https://github.com/espressif/esp-idf/issues/12231
2023-10-16 12:29:38 +02:00
wuzhenghui
5926116644 change(heap): fix and clean memory caps defination in memory_layout
1. move startup_stack attr from soc_memory_type_desc_t to soc_memory_region_t and
   remove unused aliased_iram field
2. all of the last level of RAM is retention dma accessible on esp32c3
3. remove esp32c2 and later chips retention dma accessible memory caps
4. allow allocate memory from RTC_RAM with MALLOC_CAP_EXEC cap
2023-09-28 07:53:20 +00:00
Chen Yudong
2e11919f70 fix(ci): change build-test-rules files folder 2023-09-20 19:17:06 +08:00
Armando
712c0c0075 feat(psram): esp32p4 psram device driver support 2023-08-28 14:14:58 +08:00
Armando
7dbd3f6909 feat(ci): Enable p4 example, test_apps and unit tests CI build 2023-08-24 12:51:19 +08:00
Armando
706d684418 feat(esp32p4): introduced new target esp32p4, supported hello_world 2023-08-09 19:33:25 +08:00
morris
956ec54aed fix(heap): use _SAFE version of critical section
because we allow to call malloc and free in an ISR context
2023-08-03 12:02:09 +08:00
Marius Vikhammer
39b9113f13 Merge branch 'bugfix/heap_trace_depth_assert' into 'master'
core-systems/heap: fixed wrong error message from assert in heap trace

See merge request espressif/esp-idf!24071
2023-07-11 09:15:59 +08:00
Marius Vikhammer
b70aaa16f1 Merge branch 'contrib/github_pr_11509' into 'master'
[Heap Trace Standalone] do not allocate memory until init is called (GitHub PR)

Closes IDFGH-10247

See merge request espressif/esp-idf!23958
2023-07-10 19:59:00 +08:00
Marius Vikhammer
450618c39d docs(heap): fixed typo in heap trace docs 2023-07-07 10:49:15 +08:00
morris
f0c07f82b5 fix(test): check call graph for hal component 2023-07-05 09:09:01 +08:00
Zim Kalinowski
2b4137d212 heap: enable qemu tests 2023-06-20 16:03:41 +02:00
Konstantin Kondrashov
c350c3c504 Merge branch 'feature/cleanup_wrong_log_use' into 'master'
all: Removes unnecessary newline character in logs

Closes IDFGH-10197

See merge request espressif/esp-idf!24131
2023-06-15 21:49:49 +08:00
Armando (Dou Yiwen)
17d6768e65 Merge branch 'feature/add_target_esp32p4' into 'master'
esp32p4: introduce the target

See merge request espressif/esp-idf!24155
2023-06-14 10:16:44 +08:00
Marius Vikhammer
86c5b437b5 Merge branch 'ci/trim_test_apps' into 'master'
core-system: limit build components for core-system test apps

Closes IDF-7092

See merge request espressif/esp-idf!24167
2023-06-13 15:31:00 +08:00
Armando
101e6a18eb esp32p4: introduce the target
Add esp32p4 target to tools and Kconfig
Create directories and files that are essential for `idf.py --preview set-target esp32p4`
2023-06-13 15:16:11 +08:00
Marius Vikhammer
6d11c37ff1 core-system: trim build components for core-system test apps 2023-06-13 09:14:42 +08:00
KonstantinKondrashov
e72061695e all: Removes unnecessary newline character in logs
Closes https://github.com/espressif/esp-idf/issues/11465
2023-06-09 03:31:21 +08:00
Konstantin Kondrashov
f875978fb3 Merge branch 'bugfix/incorrect_time_after_ota' into 'master'
esp_hw_support: Fix invalid system time if s_esp_rtc_time_us & s_rtc_last_ticks were moved around

Closes IDFGH-7930

See merge request espressif/esp-idf!23030
2023-06-09 00:30:52 +08:00
KonstantinKondrashov
cb6e2133f0 esp_hw_support: Fix invalid system time if s_esp_rtc_time_us & s_rtc_last_ticks were moved around
The commit fixes the case:
If variables in RTC RAM have been moved around by the linker,
they will be filled with garbage data. Any reset other than OTA would work fine
because the variables would still be initialized from the initial bootup.

So now system time will be valid even after OTA.

Closes https://github.com/espressif/esp-idf/issues/9448
2023-06-08 21:59:21 +08:00
Marius Vikhammer
c212c2b61c core-systems/heap: fixed wrong error message from assert in heap trace 2023-06-06 11:08:50 +08:00
Chip Weinberger
23cba5fe00 [Heap Trace Standalone] increase Kconfig max stack size 2023-05-31 12:41:16 +08:00
Marius Vikhammer
185d7a7bd4 heap: fixed unused variable warning 2023-05-30 11:40:17 +08:00
Chip Weinberger
fc1e8b0365 [Heap Trace Standalone] do not allocate memory until init is called 2023-05-29 00:00:15 -07:00
Marius Vikhammer
5b77f77e53 core-system/linux-target: fixed heap_caps_realloc not correctly allocating the requested number of bytes
Closes https://github.com/espressif/esp-idf/issues/11523
2023-05-29 10:09:54 +08:00
Marius Vikhammer
a300b3eb81 ci: fix invalid kconfig options in system test apps 2023-05-09 11:27:55 +08:00
laokaiyao
bf2a7b2df6 esp32h4: removed esp32h4 related codes 2023-04-23 12:03:07 +00:00
laokaiyao
b16ed57b2e esp32h4: removed esp32h4 related files 2023-04-23 12:03:07 +00:00
Guillaume Souchere
6ce1ccd964 heap: Add trace configuration to allow hash map placement in external RAM bss section when possible
- Remove the size limit for the hash_map array from the CONFIG_HEAP_TRACE_HASH_MAP_SIZE
- Add test case for heap tracing using hashmap
- Update heap_debug.rst to document the newly added configurations in the heap component

Closes https://github.com/espressif/esp-idf/issues/11172
2023-04-19 10:30:59 +02:00
Guillaume Souchere
48932ce910 Merge branch 'feature/heap-in-flash' into 'master'
heap: Add a configuration that places all the heap component in flash

Closes IDF-7143 and IDF-2853

See merge request espressif/esp-idf!23050
2023-04-13 14:38:15 +08:00
Guillaume Souchere
110aac240d heap: Add a configuration that places all the heap component and related functionalities in flash when enabled
Add test configuration to run all tests with heap component in the flash.
Add reference to this new configuration in performance section of the documentation.
2023-04-12 08:11:24 +02:00
Anton Maklakov
a1584750c8 Merge branch 'fix/heap-test-h2' into 'master'
heap: remove exclusion of esp32h2 as a target for heap pytest

Closes IDF-6880

See merge request espressif/esp-idf!23075
2023-04-12 13:43:29 +08:00
Marius Vikhammer
5cbd311ecf system: add kconfig option for using parts of SRAM1 for IRAM
Using parts of SRAM1 for IRAM allows apps with more statically allocated IRAM

Closes https://github.com/espressif/esp-idf/issues/9824
2023-04-07 07:12:58 +00:00
Jakob Hasse
dac64725af esp_system, heap: increase host test timeout for CI 2023-04-06 10:23:44 +08:00
Guillaume Souchere
c42c93758f heap: remove exclusion of esp32h2 as a target for no/light/comprehensive_poisoning tests 2023-04-05 09:19:39 +02:00
Guillaume Souchere
14fa303bbc heap: Use linked list in hashmap table to reduce collision, RAM usage and speed up the code 2023-04-05 07:44:29 +02:00
Guillaume Souchere
bdfc348ab3 heap: Add trace hash map config
- and place all added functions and vairables related to the hashmap in RAM
when the config is enabled only.

- add number of hash map entry as a Kconfig value and remove the hash map init function.
This prevents the user from allocating the hash map in flash and pass the pointer to the
init function (as the heap trace manipulate the hash map from functions placed in IRAM).

- add max linear value to the KConfig to make it configurable by the users.

- protect access to static variable "tracing"

- remove unecessary field in heap_trace_hashmap_entry_t
2023-04-05 07:44:29 +02:00
Chip Weinberger
b699033ab3 [Heap Trace] Perf: use hash map to speed up leaks mode 2023-04-05 07:44:29 +02:00
Marius Vikhammer
f627506f6b ci: update driver tests to use run_all_single_board_cases() 2023-03-28 17:05:50 +08:00
Marius Vikhammer
914bb81b95 ci: move linux host tests to test app folder 2023-03-27 17:41:44 +08:00
Guillaume Souchere
2f982107dc heap: Wrap the function hook feature under a configuration
also update heap_debug.rst and mem_alloc.rst to provide information about the feature
2023-03-23 04:50:49 +00:00
Guillaume Souchere
5a1f0cd63c heap: Make weak declaration for the alloc and free callbacks
- Fix "test get allocated size"
- Add tests for the free / alloc hooks
- Call alloc function hook on malloc/realloc/calloc base functions
- Add caps parameter to the allocation hook function
2023-03-23 04:50:49 +00:00
Chip Weinberger
1588c61c7b [Heap] new api for user heap trace callbacks 2023-03-23 04:50:49 +00:00
Zim Kalinowski
f6fc5e3c31 heap: fix and re-enable no-format warnings 2023-03-12 21:45:11 +01:00
Darian Leung
4069a62629 freertos(IDF): Remove usage of xPortGetFreeHeapSize() outside FreeRTOS
After heap_idf.c has been added (where the FreeRTOS heap is a subset of the
ESP-IDF heap), xPortGetFreeHeapSize() was updated to only returns the free
size of the FreeRTOS heap and not the entire ESP-IDF heap.

This commit replaces calls of xPortGetFreeHeapSize() with
esp_get_free_heap_size() in places outside of FreeRTOS.
2023-03-07 02:02:28 +08:00
Alexey Lapshin
4b54c49203 heap: fix gcc-12 compile errors 2023-02-22 05:33:03 +00:00
Guillaume Souchere
b1f9d7a8ee Merge branch 'feature/use-linked-list-for-heap-trace-records' into 'master'
heap: fix Leaks perf on large records by using doubly linked list

Closes IDFGH-9121 and IDF-1244

See merge request espressif/esp-idf!22009
2023-02-15 15:14:19 +08:00
Cao Sen Miao
fd3e0b0b18 esp32h2(ci): enable target test 2023-02-15 10:20:43 +08:00
Guillaume Souchere
2cf9236f6c heap trace standalone: Fix initialization of the unused linked list, update tests
- Call TAILQ_INSERT_TAIL in linked_list_setup to add unused records from the tail of the list
- Fix test "heap trace leak check" to expect that after a free, the record is zeroed instead of checking that
  the whole list of records is moved by one index in the array.
- Use esp_rom_printf() under lock instead of printf() since it does not rely on interrupts.
2023-02-14 09:48:14 +01:00
Marius Vikhammer
f9c1a2b791 Merge branch 'feature/core_c6_component_tests' into 'master'
ci: update idf-core related tests for C6

Closes IDF-6888

See merge request espressif/esp-idf!22282
2023-02-14 16:47:52 +08:00
Guillaume Souchere
a630f06132 Merge branch 'feature/verbose-alloc-failure-abort' into 'master'
heap: print size and caps when malloc fails before calling system abort

Closes IDFGH-9267

See merge request espressif/esp-idf!22192
2023-02-13 15:26:24 +08:00
Marius Vikhammer
25abc7f6d8 ci: update idf-core related tests for C6 2023-02-13 13:01:57 +08:00
Guillaume Souchere
65c946f1f2 heap: Fix priority in shex() bit shift 2023-02-10 07:43:32 +01:00
Chip Weinberger
f11256d9da [Logs] print size and caps when malloc fails 2023-02-09 06:49:34 -08:00
Guillaume Souchere
ba1d7f8d1c heap: Fix erroneous value returned by heap_caps_get_allocated_size() when poisoning is enabled
When light (or comprehensive) poisoning is enabled, the size requested by the user for allocation
is extended by a few bytes to store the canary header and footer. heap_caps_get_allocated_size() should
return the original size asked by the user (without the additional canary bytes).

test_malloc.c extended with a new test assuring that  heap_caps_get_allocated_size() returns the proper size
regardless of the degree of poisoning.
2023-02-01 10:04:38 +01:00
Guillaume Souchere
6382f28998 heap: Modify the memory type of the memory used as startup stack when memory protection is enabled
If memory protection is enabled on esp32c3 and esp32s3, we don't want to the heap component to see
the startup stack memory as D/IRAM but as DRAM only. Introduce a new type to make this possible in
the same fashion the regular D/IRAM regions are handled.
2023-01-26 10:52:34 +01:00
Guillaume Souchere
a26ecafa0d Merge branch 'fix/malloc_cap_exec_on_esp32c6' into 'master'
heap: Fix allocation issue when trying to allocate in IRAM on esp32c6

Closes IDF-6306

See merge request espressif/esp-idf!22077
2023-01-26 17:45:23 +08:00
Guillaume Souchere
fb9a7f36d4 heap: replace usage of bzero() with memset() across the component
See https://github.com/espressif/esp-idf/issues/5095
2023-01-25 09:28:56 +01:00
Guillaume Souchere
11f99ef649 heap: Add memory protection activation test
Check that when trying to allocate in IRAM with the system memory protection enabled,
null pointer is returned, or that an address in IRAM is returned if the memory protection
is disabled.
2023-01-25 07:20:57 +01:00
Guillaume Souchere
a5d6f62e7e heap: Prevent alloc from IRAM to call dram_alloc_to_iram() on esp32c6 target
Since DRAM and IRAM are superposed on esp32c6 it is not necessary to convert a freshly allocated
DRAM addr to its IRAM equivalent when MALLOC_CAP_EXEC is passed to heap_caps_malloc(). Instead,
proceed with a default allocation since the address returned by multi_heap_malloc() already belongs
to the IRAM region.

Applies for esp32c6 and every boards with superposed DRAM and IRAM addresses.
2023-01-23 07:42:20 +01:00
Chip Weinberger
a555563558 [Heap Trace Standalone] speed up sequential access 2023-01-18 08:48:28 +01:00
Cao Sen Miao
94120b82c2 esp32h2: add build test 2023-01-17 10:29:04 +08:00
Fu Hanxi
b02687a6c4 test: migrate linux target host test jobs into pytest-embedded 2023-01-16 10:25:55 +08:00
Chip Weinberger
26a5117870 [Heap Trace Standalone] fix terrible Leaks perf on large records by using doubly linked list 2023-01-13 11:43:09 +01:00
Martin Vychodil
1da27eb6fe Merge branch 'feature/storage_esp_partition_linux' into 'master'
esp_partition: added missing functions for linux target

See merge request espressif/esp-idf!21762
2023-01-13 02:04:54 +08:00
Chip Weinberger
8b2b430d68 [Heap Trace Standalone] fix terrible Leaks perf on large records by using doubly linked list 2023-01-10 21:01:11 -08:00
radek.tandler
9e191bad52 esp_partition/linux target: Add functions supporting host tests
Added statistics and wear simulation functions to support migration of
remaining storage related host tests from fixture to linux implementation
of esp_partition.
2023-01-10 14:10:14 +01:00
Guillaume Souchere
6dffac74e9 heap: Fix IRAM level 2 starting address
The IRAM level 2 address should start at the same address as DRAM level 2 and not DRAM level 3.
2023-01-06 09:30:36 +00:00
wuzhenghui
05e37ba214 esp32h2 memory: update esp32h2 memory layout 2023-01-06 05:30:24 +00:00
Guillaume Souchere
b5bacffbd3 esp_hw_support: allow writing in IRAM when CONFIG_ESP_SYSTEM_PMP_IDRAM_SPLIT is not set for esp32C2
when CONFIG_ESP_SYSTEM_PMP_IDRAM_SPLIT is not set, the PMP rule for esp32c2 shoud allow write access.
Fix esp_cpu_configure_region_protection() in cpu.c accordingly.
2023-01-02 08:41:16 +01:00
Guillaume Souchere
a218139d25 heap: RE-enable ESP32C2 test in test_apps 2023-01-02 08:40:59 +01:00
Cao Sen Miao
4713a9a7f2 ESP32H2: Introduce new chip target esp32h2, hello_world example supported 2022-12-29 12:29:14 +08:00
Guillaume Souchere
a74ae7fba7 Merge branch 'feature/heap-trace-improve-spi-ram-support' into 'master'
heap_trace: improvements to formatting, code, comments of heap trace standalone

Closes IDFGH-8699

See merge request espressif/esp-idf!21506
2022-12-28 15:07:30 +08:00
Guillaume Souchere
89c0a259ee heap-trace: Add missing tests for heap_trace_summary() and heap_trace_dump_caps()
Note: the checks for the tests marked as [test-dump] are done in the pytest function
test_heap_trace_dump() since they are relying on the content of the heap_trace_dump_caps()
output.
2022-12-27 10:24:14 +01:00
Guillaume Souchere
c39a9de344 heap_trace: unify API to pass caps param to heap_trace_dump() 2022-12-27 10:23:30 +01:00
Guillaume Souchere
c92f2394f2 Merge branch 'fix/stack-check-fail-in-flash' into 'master'
esp_system: fix placement of __stack_chk_fail from flash to RAM

See merge request espressif/esp-idf!21465
2022-12-09 18:08:57 +08:00
Chip Weinberger
9afc386dd2 [Heap Trace Standalone] improvements to formatting, code, comments 2022-12-08 00:47:02 -08:00
Guillaume Souchere
4d16f46a88 esp_system: fix placement of __stack_chk_fail from flash to RAM
When stack check is enabled, certain functions (sometimes placed in RAM)
are being decorated with stack guards and a call to __stask_chk_fail() in
case ofr stack corruption. For this reason, __stack_chk_fail() must be
placed in RAM too.

Add stack check config in heap tests on all targets to find eventual flash to RAM
calls due to stack checks when running callgraph_check.py
2022-12-07 08:49:12 +01:00
Marius Vikhammer
36cb08c379 heap: add linux target support
Add wrappers for public heap API for the linux target.
2022-12-06 10:49:59 +08:00
Song Ruo Jing
1575b9e43a ci: Disable all currently failed target tests for esp32c6 2022-11-28 12:09:08 +08:00
Guillaume Souchere
1d16ca6a26 Merge branch 'feature/lower-iram-utilization-of-heap-component' into 'master'
heap: lower the utilization of IRAM by the heap component binary

Closes IDF-2282 and IDFGH-6233

See merge request espressif/esp-idf!20926
2022-11-25 15:22:05 +08:00
C.S.M
28b2deaf24 Merge branch 'feature/add_target_esp32h2' into 'master'
ESP32-H2: Introduce new target for ESP32H2

See merge request espressif/esp-idf!21190
2022-11-24 18:31:19 +08:00
Guillaume Souchere
b496bab0f6 heap: fix linker issues and remove spi flash dependencies 2022-11-24 10:31:14 +01:00
Guillaume Souchere
e31e6d4de8 feat: remove tlsf_fls and tlsf_ffs from linker as they are inlined. 2022-11-24 10:31:14 +01:00
Guillaume Souchere
ab1bd08ab9 tools: update list of references to not include symbold used by __assert_func calls
On xtensa architecture, the call to __assert_func uses a reference to __func__ that can
sometimes be placed in flash. Since the __asert_func can be called from functions in IRAM
the check_callgraph script can report an error when checking for invalid calls from IRAM
to flash sections. However, the __asert_func prevents this scenario at runtime so the
check_callgraph script reports a 'flas positive' situation. For this reasson, all references
to __func__$x found prior to a call to __assert_func are droped in the parsing of the rtl files.
2022-11-24 10:31:14 +01:00
Guillaume Souchere
0376e6ec79 heap: add check for usage of flash content from iram
this commits:
- adds build-time test to check that no call to flash regions are done from IRAM functions

- resolves problems related to IRAM function using content in flash memory

- update heap_caps_alloc_failed to use a default function name in DRAM
  when necessary instead of creating a function name variable in DRAM for
  each call of heap_caps_alloc_failed. This allows to save some extra bytes
  in RAM.
2022-11-24 10:31:14 +01:00
Guillaume Souchere
5dc0481d21 heap: add documentation about the function placement in IRAM and its usage in ISR
This commits adds a internal.md file in the heap directory to clarify the idea behind
which functions is placed in IRAM or in flash.

A section in mem_alloc.rst documentation is added to specify which functions from the
heap component API can be used in interrupt handlers.
2022-11-24 10:31:14 +01:00
Guillaume Souchere
7fcd964b1d heap: add selective placement of function in IRAM
This commit aims to place in the IRAM section only the functions that
are relevent for performance instead of placing the entire content of
multi_heap.c, mullti_heap_poisoning.c and tlsf.c in the IRAM.
2022-11-24 10:31:14 +01:00
Cao Sen Miao
86aa4df5b5 ESP32-H2: Introduce new target for ESP32H2 2022-11-23 14:38:05 +08:00
Omar Chebib
cd21058097 C/Cxx: unify static assertions with the macro ESP_STATIC_ASSERT
Closes https://github.com/espressif/esp-idf/issues/9938
2022-11-21 16:18:08 +08:00
laokaiyao
8677216576 esp32h2: renaming esp32h2 to esp32h4 2022-11-08 17:05:33 +08:00
Song Ruo Jing
be0fdfa176 soc: Add a soc cap, SOC_CLK_RC_FAST_D256_SUPPORTED, for whether the target has the RC_FAST_D256 clock 2022-11-01 11:23:26 +08:00
Laukik Hase
d0b1b5a8aa
heap: Added note to increase INT_WDT timeout for integrity-check APIs
- When PSRAM is enabled, the heap integrity check APIs
  (`heap_caps_check_integrity` and `heap_caps_check_integrity_all`)
  take more time to execute and keep the interrupts disabled for a
  longer time.
2022-10-30 10:36:01 +05:30
Guillaume Souchere
ce895b87ab Merge branch 'feature/tlsf-dynamic-control-size' into 'master'
heap: Update to the new tlsf implementation of dynamic metadata size

See merge request espressif/esp-idf!20584
2022-10-27 13:54:45 +08:00
Guillaume Souchere
508194be50 heap: Remove size check in multi_heap.c when registering a new heap
The tlsf now checks for size validity when creating a new heap.

The check previously done in multi_heap_register_impl() is no longer
valid since the tlsf_size() is not known at this time (as the metadata
size is linked ot the size of the memory region passed as parameter
when calling tlsf_create_with_pool())

The tlsf_create_with_pool() will return a null pointer if the size
of the memory is not big enough to hold the metadata overhead and
at least a small block.

Update the test according to the changes in TLSF API
2022-10-26 18:15:42 +02:00
Guillaume Souchere
6f11d0f297 heap: Update target and host tests after incorporation of the new TLSF implementation
- update the target and host tests to consider the new TLSF api and the metadata size
  only when the target is not using the rom implementation of the TLSF
2022-10-26 18:15:40 +02:00