| Episodes | 65 |
| Mentions | 109 |
| Cited here | 38 |
| First — last | #79 — #713 |
| Top guests | Charles Aylward, Brian Amos, Elecia White |
| Related | zephyr · bluetooth · linux · wifi · microcontroller |
A real-time operating system (RTOS) is an operating system for embedded systems in which a result that the system depends on — a sensor reading or a computation — must be available within a bounded time, and its absence within that bound constitutes a failure rather than merely a delay.[584] In its minimal form, a microcontroller RTOS is a scheduler plus basic synchronisation primitives such as queues, semaphores and task notifications, without the libraries and drivers that accompany a full operating system such as Linux.[581] The category matters because it sits at the boundary between bare-metal firmware and general-purpose computing: it provides preemptive multitasking and deterministic scheduling on hardware too small for a conventional operating system, at the cost of added complexity, per-task memory and design discipline.[105][510][556]
Definition and characteristics
The defining property of a real-time system is a timing bound: when a result is relied upon, that result must be produced within a fixed amount of time, or the system has failed.[584] A general-purpose operating system does not provide this property; it will periodically go off and do something else, so a design streaming acquisition data through one must treat sustained bandwidth and buffer sizing at each end — not peak bandwidth — as the limiting factors.[135] A full Linux system is likewise non-deterministic, which means that pulling networking work onto the main processor of an embedded design leads either to an RTOS or to accepting non-deterministic timing; the alternative is a separate module with its own dedicated processor for the stack.[146]
Most microcontroller-targeted kernels are essentially a scheduler with a small set of primitives — queues, semaphores and task notifications — and ship with no libraries or drivers, which is the substantive difference from a full operating system where libraries and drivers come with the system.[581] Much of what modern embedded frameworks provide around such a kernel is a cross-hardware abstraction layer rather than real-time behaviour; the part that properly belongs to the RTOS is the kernel itself, meaning the scheduling of threads and the primitives that coordinate them.[514] For most connected products the real-time property is the least consequential thing the system provides, a view that has led practitioners to argue the category is misnamed; what such systems mostly deliver is a platform abstraction layer, where the application programs against a UART object rather than device registers and against a network connection object rather than a raw TCP/IP stack.[526]
Scheduling behaviour
Preemptive time scheduling removes the risk that adding a feature slows down or disturbs the features already present, which is why surplus processor headroom in systems such as flight controllers is spent partly on running an RTOS.[105] Unlike a fixed time-slice scheduler, which wastes the remainder of any slice a task finishes early, an RTOS scheduler does not wait for the slice to expire and moves immediately to the next task.[356] A periodic task under an RTOS does not busy-wait for its interval: it declares that it is sleeping for the period, returning the processor to the scheduler for other work, and tasks are usually partitioned by group of responsibilities rather than made extremely fine-grained.[581] Each task is given its own stack, so the memory budget of an RTOS-based design scales with the number of tasks rather than with a single call depth.[510] A common objection — that an RTOS requires dynamic memory allocation — is untrue in many cases, though the kernel does consume processor time and does add complexity.[511]
History
Early mobile robot control was built on VME-bus single-board computers running Wind River Systems’ VxWorks real-time operating system, with a tether back to a VAX in the laboratory carrying the high-level control.[334] Real-time operating systems were historically third-party products bought under licence for thousands of dollars per seat; mature open-source kernels aimed at connected devices later removed that cost from the decision.[419] The Rabbit 2000 microcontroller used a CPU architecture of its own design, written from scratch against the Z80, and shipped with an integrated real-time C operating system simple enough that a developer with no RTOS background could implement a properly scheduled system with it; the operating system was integrated directly with the C compiler so that the scheduling calls were part of the supplied toolchain, and the part carried four UARTs at a time when no other microcontroller offered that many.[419][570]
FreeRTOS carries no IP stack and no concept of networking, because it was created at a time when most RTOS-based products were unconnected devices such as medical instruments and Bluetooth headsets; kernels of a later generation were designed with internet connectivity as an integral part.[526] Repeated surveys of embedded developers have found that a large fraction of projects — in the range of forty to fifty percent — ran a homegrown real-time operating system rather than a third-party or open-source one, making the custom RTOS the single most common kind in use.[370]
Kernels and ecosystems
Ownership of the major microcontroller kernels has consolidated into large software companies: FreeRTOS is held by Amazon, ThreadX by Microsoft, and Zephyr is stewarded by the Linux Foundation.[545] As Wind River’s attention moved toward embedded Linux and larger embedded processing, the volume share of the microcontroller RTOS space passed to free kernels such as FreeRTOS.[489]
Kernel and ecosystem are separate axes. FreeRTOS is a kernel on top of which vendors build ecosystems — Espressif’s ESP-IDF, the SDK for the ESP32, is built around an RTOS port extended to run multi-core, and that kernel forms the base of the whole SDK — while Zephyr supplies both a kernel with scheduler and a vendor-neutral set of subsystems, so the two are comparable as kernels but reached from different directions.[359][675] A cross-vendor RTOS is technically the same object as any other — a scheduler, threads, queues, semaphores and mutexes — but its value lies mainly in the ecosystem: shared drivers for networking and radio are written centrally while each silicon vendor supplies a compatibility layer, so the same kernel and application code retarget across chipsets.[622] There is also a distinction between the kernel and the distribution built around it: many projects need only the kernel, and the reason they reach for one at all is to move away from a superloop.[653] Even a Cortex-M0 with about two kilobytes of RAM can run such a kernel and use its scheduler to sequence tasks in a timely fashion, though it will not drive a graphical interface; on processors smaller than that, a superloop remains the right choice.[653]
Automotive real-time operating systems are built around basic tasks — run-to-completion tasks kicked off on a fixed period such as every ten milliseconds — and nearly all real-time processing in a vehicle can be expressed that way.[634] The case for a standardised automotive kernel rests on ease of use, speed to market and the support of many vendors, tools and performance tests; nothing in it is specific to vehicles, and it suits any bare-metal problem up to the point where file systems and mass storage push a design toward real-time Linux.[634] A general-purpose cross-vendor RTOS carries a substantially larger footprint than an automotive kernel; it can be trimmed heavily, but that optimisation is left to the user.[634]
Selection criteria
The reliable indicators that a design wants an RTOS are a user interface — particularly an on-board graphical one — and extensive communications capability: network connectivity, USB, or heavy use of SPI and UART.[581] The first question, however, is whether a state machine would meet the requirement, since for a genuinely small system it is the better option.[581] The decision is easier on a clean-sheet project than on inherited code, because knowing the minimum requirements up front, and which features are merely desirable after the first working version, bounds the problem well enough to choose an architecture.[581]
The presence of a networking stack is the customary marker for moving into RTOS territory, but it is not a requirement: a multiplayer Minecraft server supporting around eight players, with an Ethernet MAC and an SD card for chunk storage, was run on an ATmega328 with roughly 32 kB of flash and 2 kB of RAM.[637] At the other end, an RTOS on a microcontroller remains the right answer for appliance-class products: a controller with an LCD, a heater, a stirring pump and a temperature sensor is comfortably within microcontroller reach, and the presence of a touchscreen does not by itself imply a Linux-class system.[515] Graphical toolkits are a genuine deciding factor between the two: the free embedded GUI libraries are workable but unattractive and awkward, while the commercial ones that look and perform well cost on the order of ten thousand dollars in licence fees, which matters most on low-volume products.[515]
For tightly written signal-processing code in which every sample that arrives is processed and passed straight out, a superloop with a handful of functions carries no scheduling overhead and an RTOS adds nothing the application needs.[511] Similarly, much of what an RTOS supplies looks self-evidently useful — assigning priorities to button presses and screen updates so the display always refreshes — but on human timescales the difference between a thirty and a two hundred millisecond response is immaterial, so those requirements alone do not justify one.[584] Sensors requiring very tight timing can be handled by a microcontroller with an RTOS, but only a fairly powerful one; FPGAs and CPLDs suit that combination of tight timing and parallel processing more naturally.[588]
Team size is a practical criterion: at around five people and above, the common vocabulary a shared framework provides is what makes parallel work possible at speed, whereas a solo developer or very small team gains more by jettisoning that complexity.[637] One consultancy standardised on an RTOS for three reasons: starting from an RTOS costs no more development time than a state-machine approach provided it is chosen at the outset rather than switched to mid-project; established boilerplate, templates, drivers and example tasks save time against writing a state machine each time; and an RTOS shoehorns acceptably into applications where a state machine or bare metal would have been the more natural fit.[645] A bare-bones scheduler such as FreeRTOS is also well suited to teaching multithreading, because the student starts from the scheduler and adds capability, seeing each layer explicitly.[675]
Design practice
An RTOS should be treated like any other dependency and wrapped behind an interface the project controls — a mutex lock and unlock taking a handle, rather than direct calls to the kernel API — so the same application code can be built and tested on a development machine as well as on the target.[556] Using an RTOS moves work forward into design: thread priorities, the threaded architecture as a whole, and the possibility of deadlock or priority inversion all become decisions that must be made deliberately rather than emerging from the code.[556] Because no two kernels share an API, changing kernel means learning an entirely new one, even though each is individually not difficult to use.[489] Moving from a single-vendor SDK to a cross-vendor RTOS likewise trades a well-defined toolchain — where a specific debug probe and IDE can be assumed — for a Linux-like model of assembling existing pieces, which brings its own learning cost.[549]
A high-priority critical task must not wait on resources shared with lower-priority work: blocking on a mutex guarding a large block of shared memory stalls the task that has the deadline, so the critical task should run continuously and push its results out to the rest of the system through a queue.[581] Interrupt priorities should be configured so that the scheduler’s own interrupt sits below every critical interrupt; the critical handler reads the streaming sensor data unconditionally, stores it, and posts a semaphore once enough has accumulated, at which point the scheduler runs the consuming tasks.[581] Because an RTOS supplies task scheduling it is tempting to stop using hardware interrupts altogether; that is possible to program but requires great care, since the timing guarantees of a hardware interrupt are not the same as those of a scheduled task.[581]
Work that would conventionally require a hardware timer — tracking a rotary encoder or a beam interrupter, for example — can be done with kernel timing services instead, but the substitution holds only within the resolution the scheduler can deliver: intervals in the region of tenths of a second are comfortable, while microsecond timing becomes difficult and belongs back in hardware.[604]
An RTOS can be instrumented so that timing itself is observable: recording when each task was activated, how long it ran, and whether it overran its real-time constraint.[584] Reducing that data to running statistics — minimum, maximum and average runtime plus a counter of missed deadlines — keeps the instrumentation affordable while carrying most of the diagnostic value of a per-activation stream, and it is what makes timing defects visible during implementation and test.[584] Before writing code, the classes of problem present in a system — distributed-system problems, real-time problems — should be identified explicitly, because each is an established field of study whose literature has to be worked through rather than rediscovered.[584]
In a hub-and-spoke system, the peripheral nodes and end effectors need not run an RTOS themselves; concentrating it in the central computer keeps the number of RTOS-based images to be maintained down as spokes are added.[584] An alternative to reconciling hard timing with communications in one scheduler is to split them across processors: dedicating a core or a programmable real-time unit, such as those on TI parts, to the critical switching task while Linux or an RTOS runs on another core for the less critical communications work.[522] Splitting a design across a Linux core, a bare-metal or RTOS core and custom logic in FPGA fabric spends timing budget on every crossing between them, and forces decisions about buffer size, how often the Linux side services the interface, and what happens to accumulated data when a service call never arrives.[466]
On the Grbl motion-control firmware ported to the ESP32, Bart Dring found that servicing web requests on the same processor as step generation produced visible stutter under high network traffic, because network work interrupted a step in progress; the resolution was to confine the motion code to one core with a single primary thread plus interrupts and avoid the RTOS there, even though the vendor API expects RTOS use throughout.[438] In power-conversion work, Fredrik Kensander’s controller expresses its scheduling entirely in interrupt priorities: interrupts at one kilohertz, one hundred hertz and ten hertz handle different work with the faster ones prioritised, a thirty-seven kilohertz interrupt in the power module must always be served, and data crosses between levels through global variables — a set point arriving over CAN or an analogue input is stored in a global that the fast interrupt reads when it fires.[522] Where that structure already meets the requirement, adding an RTOS offers no immediate benefit.[522]
Debugging changes the character of RTOS work: without step debugging the developer is left inferring behaviour from print messages and assuming the rest is working, and getting a debugger attached is the point at which that guesswork stops.[589] One connected sensor card was built on an STM32L4R5 — 640 kB of RAM and 2 MB of flash — running a simple RTOS, after the system was mocked up at a high level on a Raspberry Pi or BeagleBone first.[603]
Failure modes and limitations
The point at which a program outgrows a superloop is recognisable in its structure: a long-running computation has to be interrupted repeatedly to check unrelated conditions, such as whether a buffer is overflowing, so those checks end up sprinkled through the routine.[581] The characteristic defects in RTOS work are stack overflows and heap overflows, though a finite state machine approach produces analogous problems, so they are not by themselves an argument against the kernel.[645] A common late-stage failure is a system that almost works while the per-task stack sizes are still being balanced against one another, which is burdensome to resolve; the price of an RTOS framework is debuggability.[637]
Running an RTOS does not by itself make a design low power; consumption is determined by whether the system actually enters sleep, and treating the operating system as a substitute for deliberate power management is a mistake.[389] Documentation written at the level of introductory sketches is a further hazard when porting to a multi-core RTOS-based part: the public API functions may be listed while what is safe to call from an interrupt, what works under the RTOS and what does not, and the behaviour behind the API remain undocumented.[438] Writing an in-house RTOS is straightforward at the outset; the cost appears in continuing support years later, and in the fact that every subsequent addition must be integrated into a private code base rather than taken from a shared ecosystem.[370] A team that had run FreeRTOS for over two years declined to move to Zephyr on two grounds — keeping a single kernel across all projects, and the complexity of work outside Zephyr’s well-trodden domain such as ultra-low power on an undeveloped platform — while accepting the corresponding cost of FreeRTOS, which is that it ships no drivers and each must be written.[614]
Practitioner positions and alternatives
Elecia White, whose embedded systems book deliberately omits the topic, removes an RTOS wherever possible in favour of bare metal or a small scheduler with message passing; in robotics and in audio and visual effect systems, which must respond quickly, she judges the latency an RTOS introduces not worth its benefits, and where one is used the developer has to stay aware of what the scheduler is doing at any moment.[187] Kevin Cappuccio replaces kernel primitives with his own — writing a private mutex rather than using the supplied one — on the grounds of not trusting code whose internals he has not examined.[689]
Much RTOS-like behaviour is available without a task-switching kernel: a conventional superloop can set flags as events arrive and test those flags in the intended priority order, or queue the events and service the queue, which is adequate wherever the processor has enough headroom that timing is not tight.[584] Event-driven logic written as nested conditionals degenerates quickly, which is the argument for treating everything as a state machine from the outset: a state machine defines the device by a single state at any instant, making the environment exactly reproducible for testing and quality assurance.[711] Threads are then not the starting point for a design but a way to express behaviour that is awkward inside a pure state machine — a heartbeat LED that must keep running regardless of the current state is the canonical example, belonging in a thread rather than being replicated in every state.[711]