Commit Graph

371 Commits

Author SHA1 Message Date
Angus Gratton
5f1844a0c8 Merge branch 'bugfix/heap_poison_swap_check' into 'master'
heap: When verifying & swapping a fill pattern, stop swapping if verification fails

See merge request idf/esp-idf!2890
2018-09-03 11:19:16 +08:00
Hrishikesh Dhayagude
4abe47437f components/bt: Fix broken API esp_bt_mem_release() for parameter ESP_BT_MODE_BTDM
Problem:
The new API esp_bt_mem_release() that was added freed BTDM data to heap from esp_bt_controller_mem_release().
Now with the BT memory optimization commit ee787085f9,
the BTDM data is optimized and reduced to only 32 bytes which is not sufficient amount to be added to heap.
So, using the API leads to assert saying that the region is too small.

Solution:
Modify heap_caps_add_region_with_caps to return ESP_ERR_INVALID_SIZE in case the range is too small to create a new heap.
Do not assert if return value is ESP_ERR_INVALID_SIZE

This also fixes using API esp_bt_controller_mem_release() with ESP_BT_MODE_BTDM

Signed-off-by: Hrishikesh Dhayagude <hrishi@espressif.com>
2018-08-31 14:28:23 +00:00
Angus Gratton
a9c4ed7139 Merge branch 'master' into feature/cmake 2018-08-30 18:51:01 +08:00
Angus Gratton
776c57e0ac heap: When adding a new heap, allocate its metadata block from internal memory only 2018-08-30 06:18:18 +00:00
Ivan Grokhotkov
ed546797d0 heap: test: don’t warn about oversized mallocs 2018-08-29 14:40:28 +08:00
Angus Gratton
ff2404a272 Merge branch 'master' into feature/cmake 2018-08-16 17:14:17 +10:00
Angus Gratton
383a3f788d Merge branch 'doc/heap_tracing' into 'master'
heap docs: Fix some bad links & errors in the heap docs

See merge request idf/esp-idf!2942
2018-08-10 16:46:52 +08:00
Angus Gratton
c9cbc6b30c heap docs: Fix some bad links & errors in the heap docs
Also add a note about printf() sometimes causing on-demand allocations.
2018-08-10 15:54:58 +10:00
Angus Gratton
a5ae5ac4b3 soc: Allow components to reserve fixed memory ranges that they need
No longer necessary to keep all reserved addresses in 'soc'.

Means 'soc' does not need to know about 'bt', for example.

Also means that Bluetooth can be enabled in config without any memory being reserved for BT
controller. Only if code calling the BT controller is linked in, will this memory be reserved...
2018-08-06 01:37:55 +00:00
Angus Gratton
8cbd12b9b7 heap: When verifying & swapping a fill pattern, stop swapping if verification fails
Avoids situation where data has been overwritten during any post-mortem analysis
(core dump, gdbstub, etc).

As reported via forum https://esp32.com/viewtopic.php?f=2&t=6471&p=27790
2018-07-31 14:53:24 +10:00
Angus Gratton
ae59d7522a heap: heap_caps_malloc(..., MALLOC_CAP_32_BIT) should always align a 32-bit aligned size
Fixes crash in heap poisoning code if a non-32bit-aligned size is specified.
2018-07-31 14:29:31 +10:00
Angus Gratton
bfc6bb97c0 heap: Fix heap metadata test to account for background memory allocations 2018-07-30 11:41:30 +10:00
Ivan Grokhotkov
9a2583122c heap/tests: make IRAM allocation size divisible by 4
heap_caps_malloc will fail to poison a block in IRAM with size not
divisible by 4. The proper fix will be to make poisoning code
smarter, or to disallow allocations from IRAM with size not aligned
by 4.
2018-07-25 09:45:39 +03:00
Ivan Grokhotkov
a981e73e22 heap: move get_all_caps to IRAM, used in unit test 2018-07-24 09:56:40 +03:00
Ivan Grokhotkov
7a154d6a4e heap: fix unit test for the case when less than 10k of IRAM is available 2018-07-24 09:56:40 +03:00
Konstantin Kondrashov
899fa2dc55 assert: Fix. Move useful functions from wrapped assert functions
Moved useful functions from wrapped assert functions, because option `CONFIG_OPTIMIZATION_ASSERTIONS_DISABLED=y` will remove this functions.

Closes https://github.com/espressif/esp-idf/issues/2068
2018-06-22 15:17:55 +05:00
Angus Gratton
fd88d6d89c heap: Add integer overflow checks 2018-06-19 23:31:37 +00:00
Angus Gratton
6b9784cc45 Merge branch 'master' into feature/cmake_update 2018-05-31 14:46:23 +10:00
Angus Gratton
3fe862dacb test_multi_heap_host: Remove race conditions from parallel "make clean test" 2018-05-22 12:09:30 +10:00
Angus Gratton
68282cf80b heap/cmake: Fix linking error when heap tracing is enabled 2018-05-01 17:27:39 +10:00
Angus Gratton
1cb5712463 cmake: Add component dependency support
Components should set the COMPONENT_REQUIRES & COMPONENT_PRIVATE_REQUIRES variables to define their
requirements.
2018-04-30 09:59:20 +10:00
Angus Gratton
88df8fd293 cmake: Use cmake_lint project, tidy up all CMake source files 2018-04-30 09:59:20 +10:00
Angus Gratton
4f1a856dbf cmake: Remove defaults for COMPONENT_SRCDIRS, COMPONENT_SRCS, COMPONENT_ADD_INCLUDEDIRS
* Philosophical: "explicit is better than implicit".
* Practical: Allows useful errors if invalid directories given in components as the defaults aren't
  always used. Also trims the -I path from a number of components that have no actual include
  directory.
* Simplifies knowing which components will be header-only and which won't
2018-04-30 09:59:20 +10:00
Angus Gratton
c671a0c3eb build system: Initial cmake support, work in progress 2018-04-30 09:59:20 +10:00
Mahavir Jain
5cdd08ef5a heap: fix build failure due to missing header
Closes https://github.com/espressif/esp-idf/issues/1881

Signed-off-by: Mahavir Jain <mahavir@espressif.com>
2018-04-26 11:08:58 +05:30
Ivan Grokhotkov
573ea385b4 multi_heap: use portENTER/EXIT_CRITICAL instead of taskENTER/EXIT_CRITICAL 2018-04-19 18:28:55 +05:30
Angus Gratton
fe907bde60 Merge branch 'bugfix/heap_tracing_newlib_allocations' into 'master'
heap tracing: Fix tracing of newlib internal allocations

See merge request idf/esp-idf!1992
2018-04-18 12:07:15 +08:00
Angus Gratton
2914111c8c Merge branch 'bugfix/heap_fragmentation_on_split_pr1761' into 'master'
heap: Try to merge tail with next block when splitting

See merge request idf/esp-idf!2147
2018-04-10 12:14:57 +08:00
Anton Maklakov
046ff2e628 Make error codes looks like literals in the code of them (easy for searching). Minor fixes, typos 2018-04-09 13:54:18 +10:00
Angus Gratton
3ccb12530b heap: Change test_multi_heap_on_host multi-config to bash script not Makefile
Hopefully fix some CI building issues with parallel builds.
2018-04-03 16:34:02 +10:00
Deomid Ryabkov
4b7eb1792e Try to merge tail with next block when splitting
When splitting a memory block, check if the next block is free.
If it is, then just extend it upwards instead of creating a new block.
This fixes a bug where when shrinking existing allocations would result in irreversible free space fragmentation.

When testing on the host, test all the poisoning configurations.
2018-04-03 12:30:52 +10:00
Angus Gratton
d3b8f1f927 heap tracing: Fix tracing of newlib internal allocations
Newlib internal allocations (from newlib ROM code) were not being included in the heap trace.

Ref https://github.com/espressif/esp-idf/issues/1652
2018-03-20 12:03:46 +11:00
Stephen Casner
bc2879a956 heap: Add task tracking option for heap usage monitoring
Add back a feature that was available in the old heap implementation
in release/v2.1 and earlier: keep track of which task allocates each
block from the heap. The task handle is conditionally added as
another word in the heap poisoning header under this configuration
option CONFIG_HEAP_TASK_TRACKING.

To allow custom monitoring and debugging code to be added, add helper
functions in multi_heap.c and multi_heap_poisoning.c to provide access
to information in the block headers.

Add esp_heap_debug_dump_totals() to monitor heap usage

esp_heap_debug_dump_totals() dumps into a user-provided data structure
a summary of the amound of heap memory in region type that is used by
each task.  Optionally it will also dump into another data structure
the metadata about each allocated block for a given list of tasks or
for all tasks (limited by available space).

Address change requests on PR #1498

This set of changes fixes the files in e3b702c to just add the
CONFIG_HEAP_TASK_TRACKING option without adding the new function
heap_caps_get_per_task_info() in case that is the only portion of the
PR that will be accepted.  Part of the change is to remove the new .c
and .h files containing that function and to remove the line to
compile it from components/heap/component.mk since it should not have
been included in e3b702c.  One or more additional commits to add the
new function will follow.

The other changes here:
- uint32_t get_all_caps() moves to heap_private.h
- replace "void* foo" with "void *foo"
- add braces around single-line "if" blocks
- replace tab characters with spaces

Address change requests on PR #1498, part 2

This set of changes fixes the files in cdf32aa to add the new function
heap_caps_get_per_task_info() with its new name and to add the line to
compile it in components/heap/component.mk.  This does not address all
the suggested changes because there are some needing further
discussion.

This commit does not include the suggested change to move the
declaration of the new function into esp_heap_caps.h because the new
function references TaskHandle_t so esp_heap_caps.h would have to
include freertos/FreeRTOS.h and freertos/task.h, but FreeRTOS.h
includes esp_heap_caps.h through two other header files which results
in compilation errors because not all of FreeRTOS.h has been read yet.

Change heap_caps_get_per_task_info() to take struct of params

In addition to moving the large number of function parameters into a
struct as the single parameter, the following changes were made:

- Clear out the totals for any prepopulated tasks so the app code
  doesn't have to do it.

- Rather than partitioning the per-task totals into a predetermined
  set of heap capabilities, take a list of <caps,mask> pairs to
  compare the caps to the heap capabilities as masked.  This lets the
  caller configure the desired partitioning, or none.

- Allow the totals array pointer or the blocks array pointer to be
  NULL to indicate not to collect that part of the information.

- In addition to returning the total space allocated by each task,
  return the number of blocks allocated by each task.

- Don't need to return the heap capabilities as part of the details
  for each block since the heap region (and therefore its
  capabilities) can be determined from the block address.

- Renamed heap_task_info.h to esp_heap_task_info.h to fit the naming
  convention, and renamed the structs for totals and block details to
  better fit the revised function name.

- Provide full Doxygen commenting for the function and parameter
  structs.

Add copyright header to new files

Merges https://github.com/espressif/esp-idf/pull/1498
2018-02-20 10:32:06 +11:00
Stephen Casner
a76d5a33c9 heap: Correct the sense of a comment 2018-02-19 09:42:40 +11:00
Angus Gratton
b7fc067c8c heap: Fix bug when realloc moves data between heaps
When realloc-ing to a smaller buffer size which ends up allocated in a different heap, the heap
structure is corrupted. This can only happen:

* If heap checking is Comprehensive (meaning buffers are never shrunk in place) and the heap the buffer was originally allocated in is full.
* Calling heap_caps_realloc() to deliberately move a buffer to a different capabilities type, and shrink it at the same time.

Probable fix for https://github.com/espressif/esp-idf/issues/1582
Probably the same issue:
https://www.esp32.com/viewtopic.php?f=2&t=4583
https://www.esp32.com/viewtopic.php?f=13&t=3717
2018-02-09 16:10:52 +08:00
Kewal
ae8a0bbf56 fix typo for heap cap free size 2017-12-14 10:39:26 +05:30
Mahavir Jain
18553c451a heap_trace: fix bug in realloc for copying trace record
Closes https://github.com/espressif/esp-idf/issues/1354

Signed-off-by: Mahavir Jain <mahavir@espressif.com>
2017-12-07 18:20:23 +05:30
Ivan Grokhotkov
5fe91ae7cd Merge branch 'bugfix/cpp_guards' into 'master'
Add C++ guards to freertos/ringbuf.h, heap headers

See merge request !1552
2017-11-22 22:26:26 +08:00
Angus Gratton
a556e1c782 heap: Add C++ header guards to heap headers
Closes https://github.com/espressif/esp-idf/issues/1195
2017-11-22 10:08:50 +11:00
Angus Gratton
7a924bd85a spi_flash: Expose an accessor the current SPI flash guard functions
Change places which uses g_flash_guard_default_ops to use this. Probably exact same data, but a bit
cleaner.
2017-11-20 15:54:31 +11:00
Tian Hao
e7a9ddcf72 component/heap : fix heap_region_add check bug 2017-11-16 11:10:13 +08:00
Darian Leung
fff482acd9 freertos/add C++ include guards for esp_heap_caps.h 2017-11-10 12:01:30 +08:00
Anton Maklakov
bb4838314c doc: Add internal links for chapters and fix typos
Used 'git grep -i -E "see *\`[^\`]+\`[^_]"' to find the invalid links
2017-11-09 11:45:13 +08:00
Ivan Grokhotkov
dbebece1d2 Merge branch 'feature/unit-test-configs' into 'master'
unit-test-app: add support for testing multiple configurations

See merge request !1249
2017-10-20 14:03:53 +08:00
Angus Gratton
47aaf402b8 heap: Add heap_caps_dump() / heap_caps_dump_all() functions
Dump the structure of the heap for debugging purposes.
2017-10-20 11:38:24 +08:00
Ivan Grokhotkov
90bbcbcdc0 unit tests: fix warnings, build with -Werror
- libsodium: silence warnings
- unit tests: fix warnings
- spiram: fix warnings
- ringbuf test: enable by default, reduce delays
2017-10-19 21:35:23 +08:00
Ivan Grokhotkov
6cc8099610 Merge branch 'bugfix/malloc_failure' into 'master'
heap: Fix race condition causing malloc() to fail under some conditions

See merge request !1424
2017-10-19 21:30:26 +08:00
Angus Gratton
b0c5665f15 heap: Fix race condition causing malloc() to fail under some conditions
During a call to multi_heap_malloc(), if both these conditions were true:
- That heap only has one block large enough for the allocation
  (this is always the case if the heap is unfragmented).
- Another allocation is simultaneously occurring in the same heap.

... multi_heap_malloc() could incorrectly return NULL.

This caused IDF heap_caps_malloc() and malloc() to also fail, particularly
often if only one or two heaps had space for the allocation (otherwise
heap_caps_malloc() fails over to the next heap).
2017-10-19 16:05:00 +08:00
Angus Gratton
04188d8ec7 heap: Fix spurious heap_caps_check_integrity() errors in Comprehensive mode
Heap was not being locked before poisoning, so heap_caps_check_integrity()
would sometimes race with checking the poison bytes and print unnecessary
errors.

Details: https://esp32.com/viewtopic.php?f=2&t=3348&p=15732#p15732
2017-10-18 15:57:52 +08:00
XiaXiaotian
5df39cd4b6 Allocate some memories in SPIRAM first.
Try to allocate some WiFi and LWIP memories in SPIRAM first. If
    failed, try to allocate in internal RAM then.
2017-10-13 10:11:24 +08:00
Angus Gratton
dda136eab7 Merge branch 'bugfix/heap_debugging_cleanup' into 'master'
Heap debugging docs cleanup & convenience functions

See merge request !1347
2017-10-10 14:59:54 +08:00
Angus Gratton
f0d7cfdafe heap: Add new heap_caps_check_integrity_all() & heap_caps_check_integrity_addr() debugging functions
Easier to either check all heaps, or focus on checking a particular region.
2017-10-10 16:19:30 +11:00
Angus Gratton
bb8ba76604 Merge branch 'bugfix/multi_heap_get_info_typo' into 'master'
Fix multi_heap_get_info alias (from github)

See merge request !1357
2017-10-10 13:14:09 +08:00
Angus Gratton
ec85f9fc3f heap: Fix documentation references to multi_get_heap_info() 2017-10-09 14:42:31 +11:00
Jeroen Domburg
740f8a79f0 Add logic to make external RAM usable with malloc() 2017-09-28 17:17:50 +08:00
Deomid Ryabkov
882f164e7c Fix multi_heap_get_info alias 2017-09-26 15:00:47 +01:00
Angus Gratton
1773770f44 Merge branch 'feature/multi_heap_assert' into 'master'
multi_heap: Print the problem address when aborting due to heap corruption

See merge request !1277
2017-09-25 09:51:14 +08:00
Tian Hao
b54719d00f component/bt : fix bluetooth controller enable limit && release memory when de-initialize bluetooth controller
1. fix bluetooth controller enable limit
2. release memory when de-initialize bluetooth controller
3. fix heap_caps_add_region limit
2017-09-19 21:14:28 +08:00
Angus Gratton
959462ffb6 multi_heap_poisoning: Use MULTI_HEAP_STDERR_PRINTF (ets_printf) to print heap errors
Needed because normal printf() can trigger a malloc() (for standard stream locks) which
then re-triggers this check.
2017-09-18 16:54:28 +10:00
Angus Gratton
76d8190444 multi_heap: Print the problem address when aborting due to heap corruption
New multi_heap code has proven effective at aborting when buffer overruns occur,
but it's currently hard to debug the stack traces from these failures.
2017-09-18 16:54:23 +10:00
Jeroen Domburg
875ae6a134 Add option to allocate external RAM using heap_alloc_caps 2017-09-14 10:47:44 +08:00
Angus Gratton
72995bfcec doc: Add docs for heap trace & poisoning 2017-09-07 16:32:05 +10:00
Angus Gratton
1c7b8aa3a5 Heap tracing support 2017-09-07 16:32:05 +10:00
Angus Gratton
5c417963eb multi_heap: Add heap poisoning features 2017-09-07 16:32:05 +10:00
Angus Gratton
5222428dde unit tests: Check heap integrity after each test, check for obvious leaks 2017-09-07 16:32:05 +10:00
Angus Gratton
1cb0f30933 heap_caps: Add heap_caps_check_integrity() function 2017-09-07 16:32:05 +10:00
Angus Gratton
5361c08989 heap: Support adding new heap regions at runtime
To facilitate this, the list of registered heap regions is now a linked list
(allowing entries to be appended at runtime.)
2017-09-05 14:07:02 +10:00
Angus Gratton
0feb40833a heap_caps: Allow for possibility a region is too small to register a heap
May happen due to sdkconfig, static allocation of RAM.

Closes https://github.com/espressif/esp-idf/issues/802
2017-07-19 17:24:24 +10:00
Angus Gratton
ad60c30de0 heap: Rename memory "tags" to "types" to avoid confusion w/ old tag allocator API 2017-07-10 17:46:03 +08:00
Angus Gratton
71c70cb15c heap: Refactor heap regions/capabilities out of FreeRTOS
Remove tagged heap API, rename caps_xxx to heap_caps_xxx

Also includes additional heap_caps_xxx inspection functions.
2017-07-10 17:46:03 +08:00
Angus Gratton
5ee49fd311 heap: Add new multi_heap heap implementation to replace FreeRTOS-based tagged heaps 2017-07-10 17:46:03 +08:00