The following terms will be used in this document to avoid confusion between the different FreeRTOS versions currently in ESP-IDF
- SMP FreeRTOS: The SMP branch of the FreeRTOS kernel found [here](https://github.com/FreeRTOS/FreeRTOS-Kernel/tree/smp)
- IDF FreeRTOS: The version of FreeRTOS used in mainline ESP-IDF that contained custom modifications to support SMP features specific to the ESP chips.
This directory contains a copy of SMP FreeRTOS based off of upstream commit [2eff03708](https://github.com/FreeRTOS/FreeRTOS-Kernel/commit/2eff037080996e67a79e381667f0b77252e653b8)
- All changes made to the rest of ESP-IDF to support SMP FreeRTOS are wrapped in `#ifdef CONFIG_FREERTOS_SMP`. Thus SMP FreeRTOS changes should have no effect on mainline ESP-IDF if `CONFIG_FREERTOS_SMP` is disabled.
- Each core had its own suspendable scheduler (i.e., calling `vTaskSuspendAll()` on one core has no effect on the other)
- A core that calls `vTaskSuspendAll()` will only disable task switching on that core.
SMP FreeRTOS:
-`vTaskSuspendAll()` is global (no cores can switch tasks once the scheduler is disabled)
- While the scheduler is suspend, any other core that attempts to block, call FreeRTOS API, or also suspend the scheduler will result that the other core spinning with interrupts disabled (due to task lock contention)
Changes Made:
- Replaced all calls of `vTaskSuspendAll()`/`xTaskResumeAll()` in ESP-IDF with `vTaskPreemptionDisable()`/`vTaskPreemptionEnable()` when using SMP FreeRTOS
- The critical sections now **checks for state change**. This means if a task has just entered a critical section and another higher priority task can run, the critical section will exit to yield and then retry entering.
- In order to be compatible with IDF style critical sections, `portENTER_CRITICAL(...)`/`portEXIT_CRITICAL(...)` now accepts `VA_ARGS`
- If no argument exists, it calls SMP FreeRTOS critical sections `vTaskEnterCritical()`
- If an argument exists, it calls IDF FreeRTOS critical sections (where the argument is the spinlock)
Todo:
- Assess how using a giant lock (i.e., SMP FreeRTOS critical sections) will affect performance on time sensitive components of ESP-IDF (e.g., BT, WiFi)
- If performance is acceptable, replace all critical sections in ESP-IDF with SMP FreeRTOS critical sections
- Else, refactor IDF FreeRTOS critical sections to a separate API, and have IDF components call those instead.
## Idle Tasks
IDF FreeRTOS:
- Only uses a single idle task function `prvIdleTask()`
- An idle task is created for each core and pinned to that core
-`prvIdleTask()` calls a custom `esp_vApplicationIdleHook()`, mostly used to feed the task watchdog
- IDF adds a Thread Local Storage Pointer (TLSP) Deletion Callback that is called on task deletion (i.e., can be called from the idle task).
- There are now two types of idle task functions. The `prvIdleTask()` and `prvMinimalIdleTask()`
-`prvMinimalIdleTask()` simply calls the `vApplicationMinimalIdleHook()`
-`prvIdleTask()` calls `prvCheckTasksWaitingTermination()`, `vApplicationIdleHook()`, `vApplicationMinimalIdleHook()`, and handles tickless idle.
- On an N core build, one `prvIdleTask()` task is created and N-1 `prvMinimalIdleTask()` tasks are created.
- The created idle tasks are all unpinned. The idle tasks are run on a "first come first serve" basis meaning when a core goes idle, it selects whatever available idle task it can run.
- The `esp_vApplicationIdleHook()` is now called from `vApplicationMinimalIdleHook()` since every idle task calls the `vApplicationMinimalIdleHook()`.
- Since the idle tasks are unpinned, the task WDT has been updated to use the "User" feature. Thus, feeding the task watchdog now tracks which "core" has fed the task WDT instead of which specific idle task has fed.
- Since `prvIdleTask()` is solely responsible for calling `prvCheckTasksWaitingTermination()` but can run on any core, multiple IDF cleanup routines are now routed through `portCLEAN_UP_TCB()`
- FPU registers of a task are now cleaned up via `portCLEAN_UP_TCB() -> vPortCleanUpCoprocArea()` and can clean FPU save areas across cores.
- TLSP Deletion callbacks are now run via `portCLEAN_UP_TCB() -> vPortTLSPointersDelCb()`
- SMP FreeRTOS requires that `xTaskIncrementTick()` only be called from one core (core 0 in our case). That core's tick interrupt is solely responsible for the following via its call to `xTaskIncrementTick()`:
- See if we can find another way to implement the interrupt watchdog. Requiring each core to interrupt just to feed the watchdog loses the benefit of having core 0 handling the time slicing of all cores.
- Moved their definition/declaration to `freertos_tasks_c_additions.h`/`idf_additions.h` respectively. Users should `#include "freertos/idf_additions.h` manually so that we can keep the SMP FreeRTOS API clean.
- The `xTaskCreate...AffinitySet()` API has been added in order to apply a core affinity on task creation.
-`xTaskCreate...PinnedToCore()` now just wrap the `xTaskCreate...AffinitySet()` equivalent functions and handle the conversion of a core ID into a core affinity mask.
-`xTaskCreate...PinnedToCore()` is now provided via `idf_additions.h`
- If TLSP deletion callbacks are used, `configNUM_THREAD_LOCAL_STORAGE_POINTERS` will be doubled (in order to store the callback pointers in the same array as the TLSPs themselves)
-`vTaskSetThreadLocalStoragePointerAndDelCallback()` moved to `freertos_tasks_c_additions.h`/`idf_additions.h`
- Deletion callbacks invoked from the main idle task via `portCLEAN_UP_TCB()`
