The original FreeRTOS (hereinafter referred to as Vanilla FreeRTOS) is a compact and efficient real-time operating system supported on many single-core MCUs and SoCs. However, to support numerous dual-core ESP targets, such as ESP32, ESP32-S3, and ESP32-P4, ESP-IDF provides an implementation of FreeRTOS with dual-core symmetric multiprocessing (SMP) capabilities (hereinafter referred to as ESP-IDF FreeRTOS).
ESP-IDF FreeRTOS is based on Vanilla FreeRTOS v10.5.1 but contains significant modifications to both API and kernel behavior in order to support dual-core SMP. This document describes the API and behavioral differences between Vanilla FreeRTOS and ESP-IDF FreeRTOS.
This document assumes that the reader has a requisite understanding of Vanilla FreeRTOS, i.e., its features, behavior, and API usage. Refer to the `Vanilla FreeRTOS documentation <https://www.freertos.org/index.html>`_ for more details.
ESP-IDF FreeRTOS can be built for a single core by enabling the :ref:`CONFIG_FREERTOS_UNICORE` configuration option. ESP targets that are single core always have the :ref:`CONFIG_FREERTOS_UNICORE` option enabled. However, note that building with :ref:`CONFIG_FREERTOS_UNICORE` enabled does not equate to building with Vanilla FreeRTOS, as some of the behavioral and API changes of ESP-IDF are still present. For more details, see :ref:`freertos-smp-single-core`.
Symmetric multiprocessing is a computing architecture where two or more identical CPUs (cores) are connected to a single shared main memory and controlled by a single operating system. In general, an SMP system:
- presents an identical view of memory to each core. Thus, a piece of code that accesses a particular memory address has the same effect regardless of which core it runs on.
Although an SMP system allows threads to switch cores, there are scenarios where a thread must or should only run on a particular core. Therefore, threads in an SMP system also have a core affinity that specifies which particular core the thread is allowed to run on.
ESP targets such as ESP32, ESP32-S3, and ESP32-P4 are dual-core SMP SoCs. These targets have the following hardware features that make them SMP-capable:
CPU0 is also known as Protocol CPU or ``PRO_CPU`` and CPU1 is also known as Application CPU or ``APP_CPU``. The ``PRO_CPU`` and ``APP_CPU`` aliases for CPU0 and CPU1 exist in ESP-IDF as they reflect how typical ESP-IDF applications utilize the two CPUs. Typically, the tasks responsible for handling wireless networking (e.g., Wi-Fi or Bluetooth) are pinned to CPU0, thus the name ``PRO_CPU``; whereas the tasks handling the remainder of the application are pinned to CPU1, thus the name ``APP_CPU``.
However, in an SMP system, tasks need to be assigned a particular affinity. Therefore, ESP-IDF provides a ``PinnedToCore`` version of Vanilla FreeRTOS's task creation functions:
-:cpp:func:`xTaskCreatePinnedToCore` creates a task with a particular core affinity. The task's memory is dynamically allocated.
-:cpp:func:`xTaskCreateStaticPinnedToCore` creates a task with a particular core affinity. The task's memory is statically allocated, i.e., provided by the user.
The ``PinnedToCore`` versions of the task creation function API differ from their vanilla counterparts by having an extra ``xCoreID`` parameter that is used to specify the created task's core affinity. The valid values for core affinity are:
Note that ESP-IDF FreeRTOS still supports the vanilla versions of the task creation functions. However, these standard functions have been modified to essentially invoke their respective ``PinnedToCore`` counterparts while setting the core affinity to ``tskNO_AFFINITY``.
ESP-IDF FreeRTOS also changes the units of ``ulStackDepth`` in the task creation functions. Task stack sizes in Vanilla FreeRTOS are specified in a number of words, whereas in ESP-IDF FreeRTOS, the task stack sizes are specified in bytes.
Task deletion in Vanilla FreeRTOS is called via :cpp:func:`vTaskDelete`. The function allows deletion of another task or the currently running task if the provided task handle is ``NULL``. The actual freeing of the task's memory is sometimes delegated to the idle task if the task being deleted is the currently running task.
ESP-IDF FreeRTOS provides the same :cpp:func:`vTaskDelete` function. However, due to the dual-core nature, there are some behavioral differences when calling :cpp:func:`vTaskDelete` in ESP-IDF FreeRTOS:
- When deleting a task that is currently running on the other core, a yield is triggered on the other core, and the task's memory is freed by one of the idle tasks.
- A deleted task's memory is freed immediately if it is not running on either core.
Please avoid deleting a task that is running on another core as it is difficult to determine what the task is performing, which may lead to unpredictable behavior such as:
- Each task is given a constant priority upon creation. The scheduler executes the highest priority ready-state task.
- The scheduler can switch execution to another task without the cooperation of the currently running task.
- The scheduler periodically switches execution between ready-state tasks of the same priority in a round-robin fashion. Time slicing is governed by a tick interrupt.
The ESP-IDF FreeRTOS scheduler supports the same scheduling features, i.e., Fixed Priority, Preemption, and Time Slicing, albeit with some small behavioral differences.
In Vanilla FreeRTOS, when the scheduler selects a new task to run, it always selects the current highest priority ready-state task. In ESP-IDF FreeRTOS, each core independently schedules tasks to run. When a particular core selects a task, the core will select the highest priority ready-state task that can be run by the core. A task can be run by the core if:
However, please do not assume that the two highest priority ready-state tasks are always run by the scheduler, as a task's core affinity must also be accounted for. For example, given the following tasks:
The resulting schedule will have Task A running on CPU0 and Task C running on CPU1. Task B is not run even though it is the second-highest priority task.
In Vanilla FreeRTOS, the scheduler can preempt the currently running task if a higher priority task becomes ready to execute. Likewise in ESP-IDF FreeRTOS, each core can be individually preempted by the scheduler if the scheduler determines that a higher-priority task can run on that core.
However, there are some instances where a higher-priority task that becomes ready can be run on multiple cores. In this case, the scheduler only preempts one core. The scheduler always gives preference to the current core when multiple cores can be preempted. In other words, if the higher priority ready task is unpinned and has a higher priority than the current priority of both cores, the scheduler will always choose to preempt the current core. For example, given the following tasks:
- Task C of priority 10 that is unpinned and was unblocked by Task B
The resulting schedule will have Task A running on CPU0 and Task C preempting Task B given that the scheduler always gives preference to the current core.
The Vanilla FreeRTOS scheduler implements time slicing, which means that if the current highest ready priority contains multiple ready tasks, the scheduler will switch between those tasks periodically in a round-robin fashion.
However, in ESP-IDF FreeRTOS, it is not possible to implement perfect Round Robin time slicing due to the fact that a particular task may not be able to run on a particular core due to the following reasons:
Therefore, when a core searches the ready-state task list for a task to run, the core may need to skip over a few tasks in the same priority list or drop to a lower priority in order to find a ready-state task that the core can run.
The ESP-IDF FreeRTOS scheduler implements a Best Effort Round Robin time slicing for ready-state tasks of the same priority by ensuring that tasks that have been selected to run are placed at the back of the list, thus giving unselected tasks a higher priority on the next scheduling iteration (i.e., the next tick interrupt or yield).
- Users cannot expect multiple ready-state tasks of the same priority to run sequentially as is the case in Vanilla FreeRTOS. As demonstrated in the example above, a core may need to skip over tasks.
In ESP-IDF FreeRTOS, each core receives a periodic interrupt and independently runs the tick interrupt. The tick interrupts on each core are of the same period but can be out of phase. However, the tick responsibilities listed above are not run by all cores:
CPU0 is solely responsible for keeping time in ESP-IDF FreeRTOS. Therefore, anything that prevents CPU0 from incrementing the tick count, such as suspending the scheduler on CPU0, will cause the entire scheduler's timekeeping to lag behind.
Vanilla FreeRTOS will implicitly create an idle task of priority 0 when the scheduler is started. The idle task runs when no other task is ready to run, and it has the following responsibilities:
- Freeing the memory of deleted tasks
- Executing the application idle hook
In ESP-IDF FreeRTOS, a separate pinned idle task is created for each core. The idle tasks on each core have the same responsibilities as their vanilla counterparts.
Vanilla FreeRTOS allows the scheduler to be suspended/resumed by calling :cpp:func:`vTaskSuspendAll` and :cpp:func:`xTaskResumeAll` respectively. While the scheduler is suspended:
In ESP-IDF FreeRTOS, suspending the scheduler across multiple cores is not possible. Therefore when :cpp:func:`vTaskSuspendAll` is called on a particular core (e.g., core A):
- If an interrupt on core A unblocks any tasks, tasks with affinity to core A will go into core A's own pending ready task list. Unpinned tasks or tasks with affinity to other cores can be scheduled on cores with the scheduler running.
- If the scheduler is suspended on all cores, tasks unblocked by an interrupt will be directed to the pending ready task lists of their pinned cores. For unpinned tasks, they will be placed in the pending ready list of the core where the interrupt occurred.
- If core A is CPU0, the tick count is frozen, and a pended tick count is incremented instead. However, the tick interrupt will still occur in order to execute the application tick hook.
Given that scheduler suspension on ESP-IDF FreeRTOS only suspends scheduling on a particular core, scheduler suspension is **NOT** a valid method ensuring mutual exclusion between tasks when accessing shared data. Users should use proper locking primitives such as mutexes or spinlocks if they require mutual exclusion.
Vanilla FreeRTOS allows interrupts to be disabled and enabled by calling :c:macro:`taskDISABLE_INTERRUPTS` and :c:macro:`taskENABLE_INTERRUPTS` respectively.
Disabling interrupts is a valid method of achieving mutual exclusion in Vanilla FreeRTOS (and single-core systems in general). However, in an SMP system, disabling interrupts is **NOT** a valid method ensuring mutual exclusion. Refer to :ref:`critical-sections` for more details.
Vanilla FreeRTOS implements critical sections by disabling interrupts, which prevents preemptive context switches and the servicing of ISRs during a critical section. Thus a task/ISR that enters a critical section is guaranteed to be the sole entity to access a shared resource. Critical sections in Vanilla FreeRTOS have the following API:
However, in an SMP system, merely disabling interrupts does not constitute a critical section as the presence of other cores means that a shared resource can still be concurrently accessed. Therefore, critical sections in ESP-IDF FreeRTOS are implemented using spinlocks. To accommodate the spinlocks, the ESP-IDF FreeRTOS critical section APIs contain an additional spinlock parameter as shown below:
The critical section API can be called recursively, i.e., nested critical sections. Entering a critical section multiple times recursively is valid so long as the critical section is exited the same number of times it was entered. However, given that critical sections can target different spinlocks, users should take care to avoid deadlocking when entering critical sections recursively.
Spinlocks can be allocated statically or dynamically. As such, macros are provided for both static and dynamic initialization of spinlocks, as demonstrated by the following code snippets.
#. The core then spins on the spinlock using an atomic compare-and-set instruction until it acquires the lock. A lock is acquired when the core is able to set the lock's owner value to the core's ID.
Given that interrupts (or interrupt nesting) are disabled during a critical section, there are multiple restrictions regarding what can be done within critical sections. During a critical section, users should keep the following restrictions and considerations in mind:
However, ESP-IDF FreeRTOS implements Lazy Context Switching for the Floating Point Unit (FPU) registers of a CPU. In other words, when a context switch occurs on a particular core (e.g., CPU0), the state of the core's FPU registers is not immediately saved to the stack of the task getting switched out (e.g., Task A). The FPU registers are left untouched until:
However, given that tasks can be unpinned and thus can be scheduled on different cores (e.g., Task A switches to CPU1), it is unfeasible to copy and restore the FPU registers across cores. Therefore, when a task utilizes FPU by using a ``float`` type in its call flow, ESP-IDF FreeRTOS will automatically pin the task to the current core it is running on. This ensures that all tasks that use FPU are always pinned to a particular core.
Furthermore, ESP-IDF FreeRTOS by default does not support the usage of FPU within an interrupt context given that the FPU register state is tied to a particular task.
ESP targets that contain an FPU do not support hardware acceleration for double precision floating point arithmetic (``double``). Instead, ``double`` is implemented via software, hence the behavioral restrictions regarding the ``float`` type do not apply to ``double``. Note that due to the lack of hardware acceleration, ``double`` operations may consume significantly more CPU time in comparison to ``float``.
Although ESP-IDF FreeRTOS is an SMP scheduler, some ESP targets are single-core (such as ESP32-S2 and ESP32-C3). When building ESP-IDF applications for these targets, ESP-IDF FreeRTOS is still used but the number of cores will be set to ``1`` (i.e., the :ref:`CONFIG_FREERTOS_UNICORE` will always be enabled for single-core targets).
For multicore targets (such as ESP32 and ESP32-S3), :ref:`CONFIG_FREERTOS_UNICORE` can also be set. This results in ESP-IDF FreeRTOS only running on CPU0, and all other cores will be inactive.
Users should bear in mind that enabling :ref:`CONFIG_FREERTOS_UNICORE`**is NOT equivalent to running Vanilla FreeRTOS**. The additional APIs of ESP-IDF FreeRTOS can still be called, and the behavior changes of ESP-IDF FreeRTOS incur a small amount of overhead even when compiled for only a single core.
