Hello and welcome to the Embedded Online Conference. My name is Miro Samek,
and in this session I'm going to talk about modern embedded software architecture and design
and how it relates to the traditional Real-Time Operating System (RTOS).
Let me begin with a quick introduction:
I've been developing embedded real-time software for many years, starting with software for nuclear
physics experiments, and then programming X-ray machines for GE Medical Systems. Then I worked
for a couple of Silicon Valley companies, where I designed software for various GPS receivers.
But perhaps the most relevant for today's presentation is my teaching on YouTube,
where I've been running a video course on "Modern Embedded Systems Programming" already since 2013.
As I obviously cannot explain every concept from the first principles, throughout this
session I will be occasionally referencing the specific video lessons from that YouTube course.
Regarding the agenda for today: I will start with
describing the main challenges of embedded software development
and I'll point out which of these general problem areas this session will address.
Next, I will briefly go over the traditional sequential programming paradigm with a "superloop"
and a conventional Real-Time Operating System (RTOS). My purpose here will be to quickly
introduce the concepts and to establish common terminology. But to avoid any confusion right from
the beginning: when I say RTOS, I specifically don’t mean embedded Windows, Linux, MacOS, or
Android. I mean a deeply embedded RTOS *kernel*, such as: FreeRTOS, ThreadX, or MicroC/OS,
suitable for single-chip microcontrollers, such as those based on the ARM Cortex-M CPU.
After this, I will explain the problems caused by blocking of RTOS threads and
the best practices of concurrent programming known as the Active Object design pattern.
I will then take a look at the implementations of active objects in deeply embedded systems.
First on top of a traditional RTOS, and then with an event-driven framework.
This will allow me to draw a contrast between the two approaches.
Next, I will talk about the challenges of event-driven programming,
and specifically the causes of architectural decay known as the "spaghetti code". Here, I will
introduce modern hierarchical state machines as the most powerful "spaghetti reducers".
I will conclude with some demonstrations of modern way of creating embedded software
based on real-time embedded frameworks, model-based design, and automatic code generation.
So, what are the main challenges of embedded software development?
Well, I would classify them into the following broad categories:
The first class of problems, which I call "getting off the ground" for a lack of a better name,
is unique to deeply embedded development. Embedded systems programming always involves
custom hardware, custom cross-development tools, and a lot of low-level code.
This class of problems is addressed by MCU vendors by providing various
"platforms"--evaluation boards, cross-development tools bundled with low-level software.
Above this, lies the next layer of problems related to concurrency and real-time performance.
Concurrency problems like race conditions, deadlocks, or missed real-time deadlines
are often intermittent and hard to reproduce. They are also notoriously difficult to isolate,
fix, and reliably test for. The best possible strategy is to use a software architecture
that inherently prevents most of the concurrency problems in the first place.
This architecture is the main subject of this presentation.
The last broad class of problems relates to the particularly convoluted code structure,
which is hard to understand and very expensive or even impossible to test.
Such "spaghetti" code, also known as the "Big Ball of Mud"
tends to get worse as bugs are fixed and new features are added. This class of problems is best
addressed by proper software design and I will discuss it in the segment about state machines.
Now, the perceptions of the main challenges in embedded programming can be very different
depending on what you're trying to accomplish.
For example, hobbyists, makers and various do-it-yourself-ers
tend to see only the "getting off the ground" issues. And that's exactly the
class of problems addressed by the "maker platforms", such as Arduino or Energia.
But, the "maker platforms" often ignore and even exacerbate the next two classes of problems,
which become challenges only after newcomers actually build increasingly advanced systems. At
this point they notice that the simple "superloop" approach has shortcomings in performing various
activities concurrently. And only when they attempt to tackle the concurrency challenge,
they notice that their code tends to degenerate into "spaghetti".
In contrast, professional embedded developers working on commercial products
tend to have the exactly inverted perceptions of the most pressing challenges.
They usually don’t struggle so much with bringing up the target hardware,
the development toolchain, or even the low-level "device drivers".
Instead they worry much more about avoiding concurrency issues (such as race conditions
or missing real-time deadlines) and even more about the proper structure of their code.
So let's review the currently dominating sequential programming approach to concurrency,
based on the venerable "superloop" and the traditional RTOS.
"Superloop", also known as "main+ISRs" or "foreground/background", is the most basic
architecture that all embedded programmers learn in the beginning of their careers.
For example, here you can see the "superloop" architecture hard-coded in the Arduino main.cpp
file, which lives inside the Arduino library. This Arduino main() function
first calls the setup() callback function once at the beginning,
and the loop() callback function continuously inside the endless for-ever "superloop".
All Arduino applications, called "sketches", inherit this basic architecture and customize
it by implementing the setup() and loop() callback functions.
For instance the basic Arduino example called Blink, implements the loop() callback as follows:
The function turns an LED on, waits for 1000 ms, turns the LED off, and waits for another 1000ms.
All this repeated for-ever results in blinking the LED.
The main characteristics of this approach is that the code constantly waits in-line
for various conditions, for example a time delay. "In-line" means that the code won't proceed until
the specified condition is met. Programming that way is called *sequential* programming.
Sequential programming is straightforward, but its biggest problem is lack of *extensibility*.
For instance, suppose that you would like to extend your basic Blink "sketch"
by adding a second LED and a push-button. The second LED should turn on immediately
after you depress the button and should turn off immediately after you release the button.
The challenge is to add the new feature without disturbing the blinking of the first LED.
The most straightforward, sequential way of coding the new feature would be as follows: wait in a
while loop for as long as the push-button is not pressed, and then turn on the second LED.
Then wait in another while loop for as long as the push-button remains depressed,
and then turn the second LED off.
By the way, there is a standard Arduino Button example,
which implements exactly this functionality, but arguably not quite correctly.
This example code reads the digital pin connected to the push-button, but does not wait. Instead,
based on the state of the button it turns the LED on or off. The incorrect part is that this
code is always messing with the LED, which ends up to be tuned either on or off tens of thousands
of times per second. The LED happens to tolerate it, but some other hardware might not. Because,
really, the job was to turn the LED *once* after the button is depressed and turn it off *once*
after the button is released. This Arduino Button example shows you that often "superloop"
code tends to do too much, endlessly and mindlessly repeating unnecessary actions.
But either way, whichever implementation of Button you take,
the problem remains that it is impossible to combine it with the existing Blink code,
without completely changing its simple sequential structure.
More specifically, the problem is responsiveness or lack thereof. The sequential Blink code takes
2 seconds no matter what. The sequential Button code is even worse, because you cannot even tell
how long it will take before the button is pressed, or released for that matter.
In software architecture, the ability to compose a system from smaller components,
without disturbing those smaller components, is called "composability".
In this sense the sequential "superloop" architecture is *NOT* composable.
And here is where the Real-Time Operating System (RTOS) comes in.
The significance of an RTOS is that it makes sequential architecture composable again,
by allowing multiple "superloops" to execute seemingly simultaneously on a single CPU.
The main job of the RTOS is to create an illusion that each such "superloop", called now a thread
or a task, has the entire CPU all to itself.
To understand how that works, let's just implement the Blink-and-Button example with an RTOS.
Here, I will use the FreeRTOS kernel running on the EFM32 board with ARM Cortex-M4
microcontroller. The board conveniently provides
two LEDs and a push-button for the Blink-Button example.
The code for blinking the first LED is now called "blinky_thread",
but as you can see, it has exactly the same structure as the Blink "superloop" from Arduino.
The RTOS-based code uses vTaskDelay() FreeRTOS service,
which internally is completely different than the polling delay() function from Arduino.
But, from your perspective as an application programmer, it behaves exactly the same. Both
these functions simply wait in-line and not return until the specified time delay elapses.
The code for handling the Button and the second LED is now called
"button_thread" and also has the same structure as the sequential code for the Arduino "superloop".
This RTOS-based code uses another RTOS-service called the semaphore to wait for the button
press and for the button release. But again, from your point of view as the application developer,
the RTOS semaphore and the polling while() loop from Arduino have exactly the same effect.
Both wait in-line and don't proceed until the button is pressed and released, respectively.
The rest of the code in the RTOS implementation is there to create the two superloops,
called tasks in FreeRTOS, and to pass the control to the FreeRTOS scheduler.
The one thing that I would like to point out here is that each RTOS thread
needs its own private stack space, which requires additional memory compared to the "superloop".
When you build the code, download it to the board,
and run it,
you can see that it blinks the LED
and also reacts to pressing and releasing the button in a timely manner.
So, indeed the RTOS allows you to run the two "superloops"
seemingly simultaneously and independently on a single CPU.
Of course, in this presentation I don't have the time to really go into the inner workings
of an RTOS, but my YouTube video course contains a whole 7-lesson segment on RTOS.
But still, today I need to briefly outline how RTOS services like vTaskDelay()
or xSemaphoreTake() work, because that's the critical mechanism by which the RTOS operates.
So, when you call the RTOS vTaskDelay() service, the RTOS triggers an interrupt that,
interestingly, does not return to the original point of preemption.
Instead the interrupt returns to *another thread* chosen by
the part of the RTOS called the scheduler. This process is called "context switch".
The thread that was switched away does not progress and has its state
saved on its private stack in RAM. Such a thread does not consume any
CPU cycles and it is in a special state called the *blocked* state.
Only after the specified delay elapses, the RTOS switches the context back
to the blocked thread, which then unblocks and continues execution.
This mechanism of efficient *blocking* of threads is at the core of the RTOS,
because it allows the RTOS to realize its two main objectives.
The first objective is to is to create an illusion that all threads run simultaneously
on a single CPU. The RTOS utilizes the blocking time of threads to make progress
in other threads. Specifically, every thread needs to block somewhere in its "superloop",
or else it will take all the CPU cycles and prevent any lower-priority threads from running.
The second RTOS objective is to support the sequential programming inside threads, which,
as you recall, is based on *waiting* for something to occur, and then handling this occurrence. The
simplistic "superloop" implements the waiting by wasting all the CPU cycles in the polling loops.
In contrast, the RTOS implements waiting with efficient *blocking*.
So, in a short summary, here are the main benefits of an RTOS:
First, an RTOS enables you to decompose your embedded software
into simpler components programmed with the "wait-respond" sequential paradigm.
The RTOS works around the main problem, which is that while waiting for one kind of event,
sequential code is unresponsive to all other events.
The RTOS cannot actually change that basic fact, but it cleverly sidesteps it, by allowing
multiple sequential threads to run seemingly simultaneously. Multiple threads can wait for
multiple events in parallel, so the application as a whole remains responsive to multiple events.
The second important benefit of an RTOS is much better use of the CPU, because an RTOS supports
efficient blocking. Specifically, while a thread is waiting, it does not waste CPU
for mindless polling, but instead RTOS can switch the CPU to perform useful work in other threads.
Also, when all threads are blocked, an RTOS schedules the special *idle thread*.
This idle thread provides a very convenient, centralized place to put the system into
a low-power sleep mode. Thus an RTOS naturally supports low-power designs.
And third, an RTOS allows you to better control the timeliness of your threads. Specifically,
techniques like Rate Monotonic Analysis (RMA) describe the specific conditions under which a set
of threads can be proven to always meet their hard real-time deadlines. RMA requires a preemptive,
priority-based scheduling, and that's why most RTOSes today provide this type of scheduler.
If you wish to learn more about RMA, please watch lesson 26 in my YouTube video course.
So, there is no question that an RTOS represents a quantum leap of improvement over the "superloop".
But, as any important idea in programming, RTOS also comes with its own set of problems.
As it turns out, most of these problems can
be ultimately traced back to the central mechanism of *blocking*.
You already saw that any synchronization of threads with events,
such as timeouts or pressing and releasing a button involves *blocking*. But blocking
leads to unresponsive threads, and, as you remember, the RTOS really cannot change that.
Consequently, the main workaround is to create more threads that can wait for more events.
But this ever growing number of threads quickly becomes very expensive and unmanageable.
But even more importantly, the new threads likely need to access the same data,
peripherals, or other resources, as the already existing threads.
And this leads to *sharing of resources* among threads.
Now, you recall that context switching in an RTOS is closely related to interrupt processing,
and in fact, an RTOS just hijacks the interrupt processing hardware already available in the CPU
to do its magic. But that means that threads inherit all the problems of interrupts,
including race conditions around any shared resources.
By the way, if you don't remember what race conditions are and how to prevent them,
I recommend lesson 20 in my YouTube course.
So, now, you need to prevent race conditions. The RTOS actually supports it
by providing an assortment of mutual exclusion mechanisms, such as mutexes. But ironically,
these mechanisms lead to even more blocking, which drives the vicious cycle ever tighter.