FreeRTOS (Supplemental Features) ================================ ESP-IDF provides multiple features to supplement the features offered by FreeRTOS. These supplemental features are available on all FreeRTOS implementations supported by ESP-IDF (i.e., ESP-IDF FreeRTOS and Amazon SMP FreeRTOS). This document describes these supplemental features and is split into the following sections: .. contents:: Contents :depth: 2 .. ---------------------------------------------------- Overview ------------------------------------------------------- Overview -------- ESP-IDF adds various new features to supplement the capabilities of FreeRTOS as follows: - **Ring buffers**: Ring buffers provide a FIFO buffer that can accept entries of arbitrary lengths. - **ESP-IDF Tick and Idle Hooks**: ESP-IDF provides multiple custom tick interrupt hooks and idle task hooks that are more numerous and more flexible when compared to FreeRTOS tick and idle hooks. - **Thread Local Storage Pointer (TLSP) Deletion Callbacks**: TLSP Deletion callbacks are run automatically when a task is deleted, thus allowing users to clean up their TLSPs automatically. - **Component Specific Properties**: Currently added only one component specific property ``ORIG_INCLUDE_PATH``. .. -------------------------------------------------- Ring Buffers ----------------------------------------------------- Ring Buffers ------------ FreeRTOS provides stream buffers and message buffers as the primary mechanisms to send arbitrarily sized data between tasks and ISRs. However, FreeRTOS stream buffers and message buffers have the following limitations: - Strictly single sender and single receiver - Data is passed by copy - Unable to reserve buffer space for a deferred send (i.e., send acquire) Therefore, ESP-IDF provides a separate ring buffer implementation to address the issues above. ESP-IDF ring buffers are strictly FIFO buffers that supports arbitrarily sized items. Ring buffers are a more memory efficient alternative to FreeRTOS queues in situations where the size of items is variable. The capacity of a ring buffer is not measured by the number of items it can store, but rather by the amount of memory used for storing items. The ring buffer provides APIs to send an item, or to allocate space for an item in the ring buffer to be filled manually by the user. For efficiency reasons, **items are always retrieved from the ring buffer by reference**. As a result, all retrieved items *must also be returned* to the ring buffer by using :cpp:func:`vRingbufferReturnItem` or :cpp:func:`vRingbufferReturnItemFromISR`, in order for them to be removed from the ring buffer completely. The ring buffers are split into the three following types: **No-Split buffers** will guarantee that an item is stored in contiguous memory and will not attempt to split an item under any circumstances. Use No-Split buffers when items must occupy contiguous memory. *Only this buffer type allows you to get the data item address and write to the item by yourself.* Refer the documentation of the functions :cpp:func:`xRingbufferSendAcquire` and :cpp:func:`xRingbufferSendComplete` for more details. **Allow-Split buffers** will allow an item to be split in two parts when wrapping around the end of the buffer if there is enough space at the tail and the head of the buffer combined to store the item. Allow-Split buffers are more memory efficient than No-Split buffers but can return an item in two parts when retrieving. **Byte buffers** do not store data as separate items. All data is stored as a sequence of bytes, and any number of bytes can be sent or retrieved each time. Use byte buffers when separate items do not need to be maintained (e.g. a byte stream). .. note:: No-Split buffers and Allow-Split buffers will always store items at 32-bit aligned addresses. Therefore, when retrieving an item, the item pointer is guaranteed to be 32-bit aligned. This is useful especially when you need to send some data to the DMA. .. note:: Each item stored in No-Split or Allow-Split buffers will **require an additional 8 bytes for a header**. Item sizes will also be rounded up to a 32-bit aligned size (multiple of 4 bytes), however the true item size is recorded within the header. The sizes of No-Split and Allow-Split buffers will also be rounded up when created. Usage ^^^^^ The following example demonstrates the usage of :cpp:func:`xRingbufferCreate` and :cpp:func:`xRingbufferSend` to create a ring buffer and then send an item to it. .. code-block:: c #include "freertos/ringbuf.h" static char tx_item[] = "test_item"; ... //Create ring buffer RingbufHandle_t buf_handle; buf_handle = xRingbufferCreate(1028, RINGBUF_TYPE_NOSPLIT); if (buf_handle == NULL) { printf("Failed to create ring buffer\n"); } //Send an item UBaseType_t res = xRingbufferSend(buf_handle, tx_item, sizeof(tx_item), pdMS_TO_TICKS(1000)); if (res != pdTRUE) { printf("Failed to send item\n"); } The following example demonstrates the usage of :cpp:func:`xRingbufferSendAcquire` and :cpp:func:`xRingbufferSendComplete` instead of :cpp:func:`xRingbufferSend` to acquire memory on the ring buffer (of type `RINGBUF_TYPE_NOSPLIT`) and then send an item to it. This adds one more step, but allows getting the address of the memory to write to, and writing to the memory yourself. .. code-block:: c #include "freertos/ringbuf.h" #include "soc/lldesc.h" typedef struct { lldesc_t dma_desc; uint8_t buf[1]; } dma_item_t; #define DMA_ITEM_SIZE(N) (sizeof(lldesc_t)+(((N)+3)&(~3))) ... //Retrieve space for DMA descriptor and corresponding data buffer //This has to be done with SendAcquire, or the address may be different when we copy dma_item_t item; UBaseType_t res = xRingbufferSendAcquire(buf_handle, &item, DMA_ITEM_SIZE(buffer_size), pdMS_TO_TICKS(1000)); if (res != pdTRUE) { printf("Failed to acquire memory for item\n"); } item->dma_desc = (lldesc_t) { .size = buffer_size, .length = buffer_size, .eof = 0, .owner = 1, .buf = &item->buf, }; //Actually send to the ring buffer for consumer to use res = xRingbufferSendComplete(buf_handle, &item); if (res != pdTRUE) { printf("Failed to send item\n"); } The following example demonstrates retrieving and returning an item from a **No-Split ring buffer** using :cpp:func:`xRingbufferReceive` and :cpp:func:`vRingbufferReturnItem` .. code-block:: c ... //Receive an item from no-split ring buffer size_t item_size; char *item = (char *)xRingbufferReceive(buf_handle, &item_size, pdMS_TO_TICKS(1000)); //Check received item if (item != NULL) { //Print item for (int i = 0; i < item_size; i++) { printf("%c", item[i]); } printf("\n"); //Return Item vRingbufferReturnItem(buf_handle, (void *)item); } else { //Failed to receive item printf("Failed to receive item\n"); } The following example demonstrates retrieving and returning an item from an **Allow-Split ring buffer** using :cpp:func:`xRingbufferReceiveSplit` and :cpp:func:`vRingbufferReturnItem` .. code-block:: c ... //Receive an item from allow-split ring buffer size_t item_size1, item_size2; char *item1, *item2; BaseType_t ret = xRingbufferReceiveSplit(buf_handle, (void **)&item1, (void **)&item2, &item_size1, &item_size2, pdMS_TO_TICKS(1000)); //Check received item if (ret == pdTRUE && item1 != NULL) { for (int i = 0; i < item_size1; i++) { printf("%c", item1[i]); } vRingbufferReturnItem(buf_handle, (void *)item1); //Check if item was split if (item2 != NULL) { for (int i = 0; i < item_size2; i++) { printf("%c", item2[i]); } vRingbufferReturnItem(buf_handle, (void *)item2); } printf("\n"); } else { //Failed to receive item printf("Failed to receive item\n"); } The following example demonstrates retrieving and returning an item from a **byte buffer** using :cpp:func:`xRingbufferReceiveUpTo` and :cpp:func:`vRingbufferReturnItem` .. code-block:: c ... //Receive data from byte buffer size_t item_size; char *item = (char *)xRingbufferReceiveUpTo(buf_handle, &item_size, pdMS_TO_TICKS(1000), sizeof(tx_item)); //Check received data if (item != NULL) { //Print item for (int i = 0; i < item_size; i++) { printf("%c", item[i]); } printf("\n"); //Return Item vRingbufferReturnItem(buf_handle, (void *)item); } else { //Failed to receive item printf("Failed to receive item\n"); } For ISR safe versions of the functions used above, call :cpp:func:`xRingbufferSendFromISR`, :cpp:func:`xRingbufferReceiveFromISR`, :cpp:func:`xRingbufferReceiveSplitFromISR`, :cpp:func:`xRingbufferReceiveUpToFromISR`, and :cpp:func:`vRingbufferReturnItemFromISR` .. note:: Two calls to RingbufferReceive[UpTo][FromISR]() are required if the bytes wraps around the end of the ring buffer. Sending to Ring Buffer ^^^^^^^^^^^^^^^^^^^^^^ The following diagrams illustrate the differences between No-Split and Allow-Split buffers as compared to byte buffers with regard to sending items/data. The diagrams assume that three items of sizes **18, 3, and 27 bytes** are sent respectively to a **buffer of 128 bytes**. .. packetdiag:: ../../../_static/diagrams/ring-buffer/ring_buffer_send_non_byte_buf.diag :caption: Sending items to No-Split or Allow-Split ring buffers :align: center For No-Split and Allow-Split buffers, a header of 8 bytes precedes every data item. Furthermore, the space occupied by each item is **rounded up to the nearest 32-bit aligned size** in order to maintain overall 32-bit alignment. However, the true size of the item is recorded inside the header which will be returned when the item is retrieved. Referring to the diagram above, the 18, 3, and 27 byte items are **rounded up to 20, 4, and 28 bytes** respectively. An 8 byte header is then added in front of each item. .. packetdiag:: ../../../_static/diagrams/ring-buffer/ring_buffer_send_byte_buf.diag :caption: Sending items to byte buffers :align: center Byte buffers treat data as a sequence of bytes and does not incur any overhead (no headers). As a result, all data sent to a byte buffer is merged into a single item. Referring to the diagram above, the 18, 3, and 27 byte items are sequentially written to the byte buffer and **merged into a single item of 48 bytes**. Using SendAcquire and SendComplete ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Items in No-Split buffers are acquired (by ``SendAcquire``) in strict FIFO order and must be sent to the buffer by ``SendComplete`` for the data to be accessible by the consumer. Multiple items can be sent or acquired without calling ``SendComplete``, and the items do not necessarily need to be completed in the order they were acquired. However, the receiving of data items must occur in FIFO order, therefore not calling ``SendComplete`` for the earliest acquired item will prevent the subsequent items from being received. The following diagrams illustrate what will happen when ``SendAcquire`` and ``SendComplete`` don't happen in the same order. At the beginning, there is already a data item of 16 bytes sent to the ring buffer. Then ``SendAcquire`` is called to acquire space of 20, 8, 24 bytes on the ring buffer. .. packetdiag:: ../../../_static/diagrams/ring-buffer/ring_buffer_send_acquire_complete.diag :caption: SendAcquire/SendComplete items in No-Split ring buffers :align: center After that, we fill (use) the buffers, and send them to the ring buffer by ``SendComplete`` in the order of 8, 24, 20. When 8 bytes and 24 bytes data are sent, the consumer still can only get the 16 bytes data item. Hence, if ``SendComplete`` is not called for the 20 bytes, it will not be available, nor will the data items following the 20 bytes item. When the 20 bytes item is finally completed, all the 3 data items can be received now, in the order of 20, 8, 24 bytes, right after the 16 bytes item existing in the buffer at the beginning. Allow-Split buffers and byte buffers do not allow using ``SendAcquire`` or ``SendComplete`` since acquired buffers are required to be complete (not wrapped). Wrap around ^^^^^^^^^^^ The following diagrams illustrate the differences between No-Split, Allow-Split, and byte buffers when a sent item requires a wrap around. The diagrams assume a buffer of **128 bytes** with **56 bytes of free space that wraps around** and a sent item of **28 bytes**. .. packetdiag:: ../../../_static/diagrams/ring-buffer/ring_buffer_wrap_no_split.diag :caption: Wrap around in No-Split buffers :align: center No-Split buffers will **only store an item in continuous free space and will not split an item under any circumstances**. When the free space at the tail of the buffer is insufficient to completely store the item and its header, the free space at the tail will be **marked as dummy data**. The buffer will then wrap around and store the item in the free space at the head of the buffer. Referring to the diagram above, the 16 bytes of free space at the tail of the buffer is insufficient to store the 28 byte item. Therefore, the 16 bytes is marked as dummy data and the item is written to the free space at the head of the buffer instead. .. packetdiag:: ../../../_static/diagrams/ring-buffer/ring_buffer_wrap_allow_split.diag :caption: Wrap around in Allow-Split buffers :align: center Allow-Split buffers will attempt to **split the item into two parts** when the free space at the tail of the buffer is insufficient to store the item data and its header. Both parts of the split item will have their own headers (therefore incurring an extra 8 bytes of overhead). Referring to the diagram above, the 16 bytes of free space at the tail of the buffer is insufficient to store the 28 byte item. Therefore, the item is split into two parts (8 and 20 bytes) and written as two parts to the buffer. .. note:: Allow-Split buffers treat both parts of the split item as two separate items, therefore call :cpp:func:`xRingbufferReceiveSplit` instead of :cpp:func:`xRingbufferReceive` to receive both parts of a split item in a thread safe manner. .. packetdiag:: ../../../_static/diagrams/ring-buffer/ring_buffer_wrap_byte_buf.diag :caption: Wrap around in byte buffers :align: center Byte buffers will **store as much data as possible into the free space at the tail of buffer**. The remaining data will then be stored in the free space at the head of the buffer. No overhead is incurred when wrapping around in byte buffers. Referring to the diagram above, the 16 bytes of free space at the tail of the buffer is insufficient to completely store the 28 bytes of data. Therefore, the 16 bytes of free space is filled with data, and the remaining 12 bytes are written to the free space at the head of the buffer. The buffer now contains data in two separate continuous parts, and each continuous part will be treated as a separate item by the byte buffer. Retrieving/Returning ^^^^^^^^^^^^^^^^^^^^ The following diagrams illustrate the differences between No-Split and Allow-Split buffers as compared to byte buffers in retrieving and returning data. .. packetdiag:: ../../../_static/diagrams/ring-buffer/ring_buffer_read_ret_non_byte_buf.diag :caption: Retrieving/Returning items in No-Split and Allow-Split ring buffers :align: center Items in No-Split buffers and Allow-Split buffers are **retrieved in strict FIFO order** and **must be returned** for the occupied space to be freed. Multiple items can be retrieved before returning, and the items do not necessarily need to be returned in the order they were retrieved. However, the freeing of space must occur in FIFO order, therefore not returning the earliest retrieved item will prevent the space of subsequent items from being freed. Referring to the diagram above, the **16, 20, and 8 byte items are retrieved in FIFO order**. However, the items are not returned in the order they were retrieved. First, the 20 byte item is returned followed by the 8 byte and the 16 byte items. The space is not freed until the first item, i.e., the 16 byte item is returned. .. packetdiag:: ../../../_static/diagrams/ring-buffer/ring_buffer_read_ret_byte_buf.diag :caption: Retrieving/Returning data in byte buffers :align: center Byte buffers **do not allow multiple retrievals before returning** (every retrieval must be followed by a return before another retrieval is permitted). When using :cpp:func:`xRingbufferReceive` or :cpp:func:`xRingbufferReceiveFromISR`, all continuous stored data will be retrieved. :cpp:func:`xRingbufferReceiveUpTo` or :cpp:func:`xRingbufferReceiveUpToFromISR` can be used to restrict the maximum number of bytes retrieved. Since every retrieval must be followed by a return, the space will be freed as soon as the data is returned. Referring to the diagram above, the 38 bytes of continuous stored data at the tail of the buffer is retrieved, returned, and freed. The next call to :cpp:func:`xRingbufferReceive` or :cpp:func:`xRingbufferReceiveFromISR` then wraps around and does the same to the 30 bytes of continuous stored data at the head of the buffer. Ring Buffers with Queue Sets ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Ring buffers can be added to FreeRTOS queue sets using :cpp:func:`xRingbufferAddToQueueSetRead` such that every time a ring buffer receives an item or data, the queue set is notified. Once added to a queue set, every attempt to retrieve an item from a ring buffer should be preceded by a call to :cpp:func:`xQueueSelectFromSet`. To check whether the selected queue set member is the ring buffer, call :cpp:func:`xRingbufferCanRead`. The following example demonstrates queue set usage with ring buffers. .. code-block:: c #include "freertos/queue.h" #include "freertos/ringbuf.h" ... //Create ring buffer and queue set RingbufHandle_t buf_handle = xRingbufferCreate(1028, RINGBUF_TYPE_NOSPLIT); QueueSetHandle_t queue_set = xQueueCreateSet(3); //Add ring buffer to queue set if (xRingbufferAddToQueueSetRead(buf_handle, queue_set) != pdTRUE) { printf("Failed to add to queue set\n"); } ... //Block on queue set QueueSetMemberHandle_t member = xQueueSelectFromSet(queue_set, pdMS_TO_TICKS(1000)); //Check if member is ring buffer if (member != NULL && xRingbufferCanRead(buf_handle, member) == pdTRUE) { //Member is ring buffer, receive item from ring buffer size_t item_size; char *item = (char *)xRingbufferReceive(buf_handle, &item_size, 0); //Handle item ... } else { ... } Ring Buffers with Static Allocation ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The :cpp:func:`xRingbufferCreateStatic` can be used to create ring buffers with specific memory requirements (such as a ring buffer being allocated in external RAM). All blocks of memory used by a ring buffer must be manually allocated beforehand then passed to the :cpp:func:`xRingbufferCreateStatic` to be initialized as a ring buffer. These blocks include the following: - The ring buffer's data structure of type :cpp:type:`StaticRingbuffer_t` - The ring buffer's storage area of size ``xBufferSize``. Note that ``xBufferSize`` must be 32-bit aligned for No-Split and Allow-Split buffers. The manner in which these blocks are allocated will depend on the users requirements (e.g. all blocks being statically declared, or dynamically allocated with specific capabilities such as external RAM). .. note:: When deleting a ring buffer created via :cpp:func:`xRingbufferCreateStatic`, the function :cpp:func:`vRingbufferDelete` will not free any of the memory blocks. This must be done manually by the user after :cpp:func:`vRingbufferDelete` is called. The code snippet below demonstrates a ring buffer being allocated entirely in external RAM. .. code-block:: c #include "freertos/ringbuf.h" #include "freertos/semphr.h" #include "esp_heap_caps.h" #define BUFFER_SIZE 400 //32-bit aligned size #define BUFFER_TYPE RINGBUF_TYPE_NOSPLIT ... //Allocate ring buffer data structure and storage area into external RAM StaticRingbuffer_t *buffer_struct = (StaticRingbuffer_t *)heap_caps_malloc(sizeof(StaticRingbuffer_t), MALLOC_CAP_SPIRAM); uint8_t *buffer_storage = (uint8_t *)heap_caps_malloc(sizeof(uint8_t)*BUFFER_SIZE, MALLOC_CAP_SPIRAM); //Create a ring buffer with manually allocated memory RingbufHandle_t handle = xRingbufferCreateStatic(BUFFER_SIZE, BUFFER_TYPE, buffer_storage, buffer_struct); ... //Delete the ring buffer after used vRingbufferDelete(handle); //Manually free all blocks of memory free(buffer_struct); free(buffer_storage); .. ------------------------------------------- ESP-IDF Tick and Idle Hooks --------------------------------------------- ESP-IDF Tick and Idle Hooks --------------------------- FreeRTOS allows applications to provide a tick hook and an idle hook at compile time: - FreeRTOS tick hook can be enabled via the :ref:`CONFIG_FREERTOS_USE_TICK_HOOK` option. The application must provide the ``void vApplicationTickHook( void )`` callback. - FreeRTOS idle hook can be enabled via the :ref:`CONFIG_FREERTOS_USE_IDLE_HOOK` option. The application must provide the ``void vApplicationIdleHook( void )`` callback. However, the FreeRTOS tick hook and idle hook have the following draw backs: - The FreeRTOS hooks are registered at compile time - Only one of each hook can be registered - On multi-core targets, the FreeRTOS hooks are symmetric, meaning each CPU's tick interrupt and idle tasks ends up calling the same hook. Therefore, ESP-IDF tick and idle hooks are provided to supplement the features of FreeRTOS tick and idle hooks. The ESP-IDF hooks have the following features: - The hooks can be registered and deregistered at run-time - Multiple hooks can be registered (with a maximum of 8 hooks of each type per CPU) - On multi-core targets, the hooks can be asymmetric, meaning different hooks can be registered to each CPU ESP-IDF hooks can be registered and deregistered using the following APIs: - For tick hooks: - Register using :cpp:func:`esp_register_freertos_tick_hook` or :cpp:func:`esp_register_freertos_tick_hook_for_cpu` - Deregister using :cpp:func:`esp_deregister_freertos_tick_hook` or :cpp:func:`esp_deregister_freertos_tick_hook_for_cpu` - For idle hooks: - Register using :cpp:func:`esp_register_freertos_idle_hook` or :cpp:func:`esp_register_freertos_idle_hook_for_cpu` - Deregister using :cpp:func:`esp_deregister_freertos_idle_hook` or :cpp:func:`esp_deregister_freertos_idle_hook_for_cpu` .. note:: The tick interrupt stays active while the cache is disabled, therefore any tick hook (FreeRTOS or ESP-IDF) functions must be placed in internal RAM. Please refer to the :ref:`SPI flash API documentation ` for more details. .. -------------------------------------------------- TLSP Callback ---------------------------------------------------- TLSP Deletion Callbacks ----------------------- Vanilla FreeRTOS provides a Thread Local Storage Pointers (TLSP) feature. These are pointers stored directly in the Task Control Block (TCB) of a particular task. TLSPs allow each task to have its own unique set of pointers to data structures. Vanilla FreeRTOS expects users to... - set a task's TLSPs by calling :cpp:func:`vTaskSetThreadLocalStoragePointer` after the task has been created. - get a task's TLSPs by calling :cpp:func:`pvTaskGetThreadLocalStoragePointer` during the task's lifetime. - free the memory pointed to by the TLSPs before the task is deleted. However, there can be instances where users may want the freeing of TLSP memory to be automatic. Therefore, ESP-IDF provides the additional feature of TLSP deletion callbacks. These user provided deletion callbacks are called automatically when a task is deleted, thus allowing the TLSP memory to be cleaned up without needing to add the cleanup logic explicitly to the code of every task. The TLSP deletion callbacks are set in a similar fashion to the TLSPs themselves. - :cpp:func:`vTaskSetThreadLocalStoragePointerAndDelCallback` sets both a particular TLSP and its associated callback. - Calling the Vanilla FreeRTOS function :cpp:func:`vTaskSetThreadLocalStoragePointer` will simply set the TLSP's associated Deletion Callback to `NULL` meaning that no callback will be called for that TLSP during task deletion. When implementing TLSP callbacks, users should note the following: - The callback **must never attempt to block or yield** and critical sections should be kept as short as possible - The callback is called shortly before a deleted task's memory is freed. Thus, the callback can either be called from :cpp:func:`vTaskDelete` itself, or from the idle task. .. ------------------------------------------ Component Specific Properties -------------------------------------------- Component Specific Properties ----------------------------- Besides standard component variables that are available with basic cmake build properties, FreeRTOS component also provides arguments (only one so far) for simpler integration with other modules: - `ORIG_INCLUDE_PATH` - contains an absolute path to freertos root include folder. Thus instead of `#include "freertos/FreeRTOS.h"` you can refer to headers directly: `#include "FreeRTOS.h"`. .. -------------------------------------------------- API Reference ---------------------------------------------------- API Reference ------------- Ring Buffer API ^^^^^^^^^^^^^^^ .. include-build-file:: inc/ringbuf.inc Hooks API ^^^^^^^^^ .. include-build-file:: inc/esp_freertos_hooks.inc