The concurrency problems are failures that appear when multiple tasks interact with the same resources or wait for each other.
So far we have seen the tools FreeRTOS offers us to bring order to chaos: Queues, Semaphores and Mutexes. They are very useful tools, but by introducing concurrency we also open the door to bugs that are quite difficult to debug.
The system doesn’t give a compilation error, not even an immediate crash. It simply stops, responds late, or becomes slow intermittently. Today we are going to talk about two classics: the Priority Inversion and the Deadlock.
Priority Inversion
This phenomenon occurs when a high-priority task is forced to wait for a low-priority task, breaking the logic of the priority system.
We already mentioned this briefly in the previous article about Mutexes, but let’s dissect it because it is important to understand the sequence of events that leads to disaster.
How it occurs
Imagine three tasks with different priorities:
- Task H (High): High priority (Critical).
- Task M (Medium): Medium priority (Processing).
- Task L (Low): Low priority (Maintenance).
And a shared resource protected by a semaphore (let’s say, an I2C bus).
- L is running (because no one else wants the CPU) and takes the semaphore for the I2C.
- At that moment, an interrupt arrives and wakes up H.
- H takes over the CPU, preempts L, and starts executing.
- H tries to use the I2C, but sees it is busy. Since it’s a protected resource, H blocks waiting for it to be released.
- The CPU goes back to L so it can finish and release the semaphore.
So far, everything is normal. H has to wait a bit for L. That’s the price of sharing resources.
The Problem
While L holds the semaphore and H is waiting… M wakes up!
- Since M has higher priority than L, the Scheduler stops L and runs M.
- M runs happily. It doesn’t need the I2C, so it continues with its own business.
- M might take 500ms to finish.
The result?
- H is waiting for L.
- L is waiting for M to release the CPU.
- In practice: H (High) is waiting for M (Medium).
We have inverted the priorities. A medium-priority task is indefinitely blocking the critical task, simply because the low-priority task holds a resource hostage and isn’t allowed to release it.
This problem almost caused the failure of NASA’s Mars Pathfinder mission in 1997. The system kept resetting itself because a weather information task (Medium) prevented a bus management task (Low) from releasing a Mutex needed by the main control task (High).
Priority Inheritance
As we saw, the solution is to use Mutexes instead of Binary Semaphores to protect resources.
FreeRTOS implements Priority Inheritance in Mutexes:
- When H blocks waiting for L, the kernel temporarily raises L’s priority to match H’s.
- Now, when M tries to run, it cannot, because L (now disguised as High priority) is more important.
- L finishes quickly, releases the Mutex, and returns to its original low priority.
- H takes over.
If you protect a resource, use xSemaphoreCreateMutex. Keep in mind that inheritance reduces inversion, but does not eliminate the waiting time nor replace temporal analysis. Designs with multiple nested mutexes require special care.
Deadlock
The Deadlock is an even worse scenario. In priority inversion the system slows down; in Deadlock, the system freezes forever.
It occurs when two (or more) tasks block each other waiting for a resource that the other holds. It is a vicious circle from which no one can escape.
How it occurs
Imagine two resources: WiFi and SD card, protected by two mutexes (mutWiFi, mutSD).
And two tasks:
- Task A: Wants to save a log to the SD and then send it via WiFi.
- Task B: Wants to download a file via WiFi and save it to the SD.
// Pseudocode for Disaster
void TaskA() {
take(mutSD); // I have the SD
// ... Context Switch right here ...
take(mutWiFi); // I try to grab the WiFi... (Waiting)
}
void TaskB() {
take(mutWiFi); // I have the WiFi
// ...
take(mutSD); // I try to grab the SD... (Waiting for Task A)
}- A takes the SD.
- The Scheduler switches to B.
- B takes the WiFi.
- B tries to take the SD. It’s held by A. B blocks.
- The Scheduler goes back to A.
- A tries to take the WiFi. It’s held by B. A blocks.
Result: A waits for B, and B waits for A. Both tasks remain in the Blocked state forever. The rest of the system might keep running (if there are other tasks), but the WiFi and SD functionality is dead.
How to Avoid Deadlock
Deadlock is a logical design error. Neither the compiler nor the operating system fixes it (FreeRTOS does not detect Deadlocks automatically). We have to fix it ourselves with good practices.
Hierarchical Order
Establish a strict design rule: “Resources are always taken in the same order.”
If we decide the order is 1. WiFi, 2. SD:
- Task B: Takes WiFi -> Takes SD. (Correct)
- Task A: Should take WiFi -> Takes SD. (Even if it only needs the SD first, if it will need WiFi later, it must follow the order or acquire them all at the start).
If both tasks try to grab the WiFi first, the second one to arrive will block at the first step, but there will be no deadly embrace because neither holds the resource the other wants yet.
Timeouts and Backoff
Don’t use portMAX_DELAY (infinite wait) for Mutexes.
if (xSemaphoreTake(mutSD, pdMS_TO_TICKS(100)) == pdTRUE) {
if (xSemaphoreTake(mutWiFi, pdMS_TO_TICKS(100)) != pdTRUE) {
// Could not grab the second resource: release the first.
xSemaphoreGive(mutSD);
vTaskDelay(pdMS_TO_TICKS(20));
} else {
// Use both resources and release them in reverse order.
xSemaphoreGive(mutWiFi);
xSemaphoreGive(mutSD);
}
}It is similar to what we humans do when we meet in a narrow hallway: we both step aside, wait a second, and one passes.
Single Owner
Another option is that a single task owns the resource. The others send operations to it via a queue and receive the result via another queue or a notification. This eliminates the need to take that mutex from multiple places.
Do not attempt to take a mutex inside a critical section. Taking a mutex can block the task, which is incompatible with having suspended scheduling or masked interrupts.
Diagnostic Checklist
When the system hangs or responds late, review this list:
- Starvation: Do you have a high-priority task in a
while(1)loop withoutvTaskDelay? - Deadlock: Do you have two tasks waiting for cross-linked resources? (Check the order of
Takeoperations). - Priority Inversion: Are you using Binary Semaphores to protect variables? (Replace them with Mutexes).
- Race Condition: Are two tasks writing to the same variable without protection?
- Stack Overflow: Have you correctly calculated the task’s memory? (Use
uxTaskGetStackHighWaterMark). - Interrupt Storm: Is your interrupt firing too fast and saturating the CPU?
- Unprotected Reentrancy: Are you calling a non-reentrant function (
strtok,printfin some implementations) from two tasks simultaneously?