heap: ported tlsf allocator into multi heap
heap_host_tests: added tlsf allocator into host test
heap_host_test: update freebytes after using free
heap_tests: tlsf now passing on host tests without poisoning
multi_heap: added support for memalign using tlsf implementation
heap_caps: removed heap_caps_aligned_free
heap/test: fixed broken aligned alloc test build
heap: added poisoning pattern when blocks are being merged
heap/tests: added timing tests for memory allocation
heap: reduced tlsf structure overhead
heap/tlsf: made all short functions inside of tlsf module as inline to improve timings
heap: moved tlsf heap routines outside of flash memory
newlib: linked multiheap memalign with newlib memalign function
heap: moved block member functions to a separate file so multi_heap can use the functions
heap/test: improved the tlsf timing test
heap/test: added memalign on aligned alloc tests
heap: moved tlsf configuration constants to a separated file
heap: added random allocations test with timings
heap: modified the calculation of heap free bytes
heap: make aligned free true deprecated functions and update their documentation
heap: add extra assert after successive mallocs on small allocation host test
heap: remove legacy aligned alloc implementation.
performance: added malloc and free time performance default values
* changing dependencies from unity->cmock
* added component.mk and Makefile.projbuild
* ignore test dir in gen_esp_err_to_name.py
* added some brief introduction of CMock in IDF
Goal is that multiple faults would be required to bypass a boot-time signature check.
- Also strengthens some address range checks for safe app memory addresses
- Change pre-enable logic to also check the bootloader signature before enabling SBV2 on ESP32
Add some additional checks for invalid sections:
- Sections only partially in DRAM or IRAM are invalid
- If a section is in D/IRAM, allow the possibility only some is in D/IRAM
- Only pass sections that are entirely in the same type of RTC memory region
Configurable option to use IRAM as byte accessible memory (in single core mode) using
load-store (non-word aligned and non-word size IRAM access specific) exception handlers.
This allows to use IRAM for use-cases where certain performance penalty
(upto 170 cpu cycles per load or store operation) is acceptable. Additional configuration
option has been provided to redirect mbedTLS specific in-out content length buffers to
IRAM (in single core mode), allows to save 20KB per TLS connection.
DISABLED_FOR_TARGETS macros are used
Partly revert "ci: disable unavailable tests for esp32s2beta"
This partly reverts commit 76a3a5fb48.
Partly revert "ci: disable UTs for esp32s2beta without runners"
This partly reverts commit eb158e9a22.
Partly revert "fix unit test and examples for s2beta"
This partly reverts commit 9baa7826be.
Partly revert "efuse: Add support for esp32s2beta"
This partly reverts commit db84ba868c.
On Xtensa, backtrace can not recover the two most significant bits of
the address, as the window call size is encoded in these bits.
Because of this, __builtin_return_address modifies these MSBs to
match those of the callee, "fixing" the address. An unfortunate side
effect is that the zero return address, which usually terminates the
backtrace, gets converted to 0x40000000. While there is a valid
instruction at this address, its occurrence in the backtrace is
highly unlikely: this is the first instruction of WindowOverflow4
vector, and IDF apps switch VECBASE to an IRAM location very early at
startup.
Do not include bootloader in flash target when secure boot is enabled.
Emit signing warning on all cases where signed apps are enabled (secure
boot and signed images)
Follow convention of capital letters for SECURE_BOOT_SIGNING_KEY
variable, since it is
relevant to other components, not just bootloader.
Pass signing key and verification key via config, not requiring
bootloader to know parent app dir.
Misc. variables name corrections
This commit refactors backtracing within the panic handler so that a common
function esp_backtrace_get_next_frame() is used iteratively to traverse a
callstack.
A esp_backtrace_print() function has also be added that allows the printing
of a backtrace at runtime. The esp_backtrace_print() function allows unity to
print the backtrace of failed test cases and jump back to the main test menu
without the need reset the chip. esp_backtrace_print() can also be used as a
debugging function by users.
- esp_stack_ptr_is_sane() moved to soc_memory_layout.h
- removed uncessary includes of "esp_debug_helpers.h"
!4452 used setting LINK_LIBRARIES and INTERFACE_LINK_LIBRARIES to link
components built under ESP-IDF build system. However, LINK_LIBRARIES does
not produce behavior same as linking PRIVATE. This MR uses the new
signature for target_link_libraries directly instead. This also moves
setting dependencies during component registration rather than after all
components have been processed.
The consequence is that internally, components have to use the new
signature form as well. This does not affect linking the components to
external targets, such as with idf_as_lib example. This only affects
linking additional libraries to ESP-IDF libraries outside component processing (after
idf_build_process), which is not even possible for CMake<v3.13 as
target_link_libraries is not valid for targets not created in current
directory. See https://cmake.org/cmake/help/v3.13/policy/CMP0079.html#policy:CMP0079
Using xxx_periph.h in whole IDF instead of xxx_reg.h, xxx_struct.h, xxx_channel.h ... .
Cleaned up header files from unnecessary headers (releated to soc/... headers).
This MR removes the common dependency from every IDF components to the SOC component.
Currently, in the ``idf_functions.cmake`` script, we include the header path of SOC component by default for all components.
But for better code organization (or maybe also benifits to the compiling speed), we may remove the dependency to SOC components for most components except the driver and kernel related components.
In CMAKE, we have two kinds of header visibilities (set by include path visibility):
(Assume component A --(depends on)--> B, B is the current component)
1. public (``COMPONENT_ADD_INCLUDEDIRS``): means this path is visible to other depending components (A) (visible to A and B)
2. private (``COMPONENT_PRIV_INCLUDEDIRS``): means this path is only visible to source files inside the component (visible to B only)
and we have two kinds of depending ways:
(Assume component A --(depends on)--> B --(depends on)--> C, B is the current component)
1. public (```COMPONENT_REQUIRES```): means B can access to public include path of C. All other components rely on you (A) will also be available for the public headers. (visible to A, B)
2. private (``COMPONENT_PRIV_REQUIRES``): means B can access to public include path of C, but don't propagate this relation to other components (A). (visible to B)
1. remove the common requirement in ``idf_functions.cmake``, this makes the SOC components invisible to all other components by default.
2. if a component (for example, DRIVER) really needs the dependency to SOC, add a private dependency to SOC for it.
3. some other components that don't really depends on the SOC may still meet some errors saying "can't find header soc/...", this is because it's depended component (DRIVER) incorrectly include the header of SOC in its public headers. Moving all this kind of #include into source files, or private headers
4. Fix the include requirements for some file which miss sufficient #include directives. (Previously they include some headers by the long long long header include link)
This is a breaking change. Previous code may depends on the long include chain.
You may need to include the following headers for some files after this commit:
- soc/soc.h
- soc/soc_memory_layout.h
- driver/gpio.h
- esp_sleep.h
The major broken include chain includes:
1. esp_system.h no longer includes esp_sleep.h. The latter includes driver/gpio.h and driver/touch_pad.h.
2. ets_sys.h no longer includes soc/soc.h
3. freertos/portmacro.h no longer includes soc/soc_memory_layout.h
some peripheral headers no longer includes their hw related headers, e.g. rom/gpio.h no longer includes soc/gpio_pins.h and soc/gpio_reg.h
BREAKING CHANGE
1. separate rom include files and linkscript to esp_rom
2. modefiy "include rom/xxx.h" to "include esp32/rom/xxx.h"
3. Forward compatible
4. update mqtt
This commit removes trailing semicolons following a while(0) from
function-like macros in IDF. This will force those macros to be called
with a semicolon when called.
* Prevents section type conflict errors if (say) const & non-const data
is put into the same section (ie with DRAM_ATTR)
* Allows linker --gc-sections to remove unused custom sections
New unity component can be used for testing other applications.
Upstream version of Unity is included as a submodule.
Utilities specific to ESP-IDF unit tests (partitions, leak checking
setup/teardown functions, etc) are kept only in unit-test-app.
Kconfig options are added to allow disabling certain Unity features.
Problem:
The new API esp_bt_mem_release() that was added freed BTDM data to heap from esp_bt_controller_mem_release().
Now with the BT memory optimization commit ee787085f9,
the BTDM data is optimized and reduced to only 32 bytes which is not sufficient amount to be added to heap.
So, using the API leads to assert saying that the region is too small.
Solution:
Modify heap_caps_add_region_with_caps to return ESP_ERR_INVALID_SIZE in case the range is too small to create a new heap.
Do not assert if return value is ESP_ERR_INVALID_SIZE
This also fixes using API esp_bt_controller_mem_release() with ESP_BT_MODE_BTDM
Signed-off-by: Hrishikesh Dhayagude <hrishi@espressif.com>
No longer necessary to keep all reserved addresses in 'soc'.
Means 'soc' does not need to know about 'bt', for example.
Also means that Bluetooth can be enabled in config without any memory being reserved for BT
controller. Only if code calling the BT controller is linked in, will this memory be reserved...
heap_caps_malloc will fail to poison a block in IRAM with size not
divisible by 4. The proper fix will be to make poisoning code
smarter, or to disallow allocations from IRAM with size not aligned
by 4.
Moved useful functions from wrapped assert functions, because option `CONFIG_OPTIMIZATION_ASSERTIONS_DISABLED=y` will remove this functions.
Closes https://github.com/espressif/esp-idf/issues/2068
* Philosophical: "explicit is better than implicit".
* Practical: Allows useful errors if invalid directories given in components as the defaults aren't
always used. Also trims the -I path from a number of components that have no actual include
directory.
* Simplifies knowing which components will be header-only and which won't
When splitting a memory block, check if the next block is free.
If it is, then just extend it upwards instead of creating a new block.
This fixes a bug where when shrinking existing allocations would result in irreversible free space fragmentation.
When testing on the host, test all the poisoning configurations.
Add back a feature that was available in the old heap implementation
in release/v2.1 and earlier: keep track of which task allocates each
block from the heap. The task handle is conditionally added as
another word in the heap poisoning header under this configuration
option CONFIG_HEAP_TASK_TRACKING.
To allow custom monitoring and debugging code to be added, add helper
functions in multi_heap.c and multi_heap_poisoning.c to provide access
to information in the block headers.
Add esp_heap_debug_dump_totals() to monitor heap usage
esp_heap_debug_dump_totals() dumps into a user-provided data structure
a summary of the amound of heap memory in region type that is used by
each task. Optionally it will also dump into another data structure
the metadata about each allocated block for a given list of tasks or
for all tasks (limited by available space).
Address change requests on PR #1498
This set of changes fixes the files in e3b702c to just add the
CONFIG_HEAP_TASK_TRACKING option without adding the new function
heap_caps_get_per_task_info() in case that is the only portion of the
PR that will be accepted. Part of the change is to remove the new .c
and .h files containing that function and to remove the line to
compile it from components/heap/component.mk since it should not have
been included in e3b702c. One or more additional commits to add the
new function will follow.
The other changes here:
- uint32_t get_all_caps() moves to heap_private.h
- replace "void* foo" with "void *foo"
- add braces around single-line "if" blocks
- replace tab characters with spaces
Address change requests on PR #1498, part 2
This set of changes fixes the files in cdf32aa to add the new function
heap_caps_get_per_task_info() with its new name and to add the line to
compile it in components/heap/component.mk. This does not address all
the suggested changes because there are some needing further
discussion.
This commit does not include the suggested change to move the
declaration of the new function into esp_heap_caps.h because the new
function references TaskHandle_t so esp_heap_caps.h would have to
include freertos/FreeRTOS.h and freertos/task.h, but FreeRTOS.h
includes esp_heap_caps.h through two other header files which results
in compilation errors because not all of FreeRTOS.h has been read yet.
Change heap_caps_get_per_task_info() to take struct of params
In addition to moving the large number of function parameters into a
struct as the single parameter, the following changes were made:
- Clear out the totals for any prepopulated tasks so the app code
doesn't have to do it.
- Rather than partitioning the per-task totals into a predetermined
set of heap capabilities, take a list of <caps,mask> pairs to
compare the caps to the heap capabilities as masked. This lets the
caller configure the desired partitioning, or none.
- Allow the totals array pointer or the blocks array pointer to be
NULL to indicate not to collect that part of the information.
- In addition to returning the total space allocated by each task,
return the number of blocks allocated by each task.
- Don't need to return the heap capabilities as part of the details
for each block since the heap region (and therefore its
capabilities) can be determined from the block address.
- Renamed heap_task_info.h to esp_heap_task_info.h to fit the naming
convention, and renamed the structs for totals and block details to
better fit the revised function name.
- Provide full Doxygen commenting for the function and parameter
structs.
Add copyright header to new files
Merges https://github.com/espressif/esp-idf/pull/1498
When realloc-ing to a smaller buffer size which ends up allocated in a different heap, the heap
structure is corrupted. This can only happen:
* If heap checking is Comprehensive (meaning buffers are never shrunk in place) and the heap the buffer was originally allocated in is full.
* Calling heap_caps_realloc() to deliberately move a buffer to a different capabilities type, and shrink it at the same time.
Probable fix for https://github.com/espressif/esp-idf/issues/1582
Probably the same issue:
https://www.esp32.com/viewtopic.php?f=2&t=4583https://www.esp32.com/viewtopic.php?f=13&t=3717
During a call to multi_heap_malloc(), if both these conditions were true:
- That heap only has one block large enough for the allocation
(this is always the case if the heap is unfragmented).
- Another allocation is simultaneously occurring in the same heap.
... multi_heap_malloc() could incorrectly return NULL.
This caused IDF heap_caps_malloc() and malloc() to also fail, particularly
often if only one or two heaps had space for the allocation (otherwise
heap_caps_malloc() fails over to the next heap).
New multi_heap code has proven effective at aborting when buffer overruns occur,
but it's currently hard to debug the stack traces from these failures.
