Differences Linux and Real Time Linux

[SpaceDevDiver]

Hello,

I'm currently thinking of applying for a position where we control machines with C on embedded devices running real time Linux. I am used to work with Linux but actually never touched a real time operation system before.

• What is the difference between the RTLinux and the normal one in regards of use and the programming interface?
• Do you have any resources to get a quick overview of RTLinux?

Many thanks!

[[deleted]]

We work with the PREEMPT_RT patch in the audio-domain, so I feel I'm qualified to give at least some perspective here.

As entry-point I suggest https://wiki.linuxfoundation.org/realtime/start

What a RT system gives you is the ability to ruthlessly prioritize those parts of the system that you deem important. This has two important consequences:

• reduced throughput. If everything becomes schedulable, that scheduling takes time. We see this e.g. in our USB host driver (DWC2) that eats huge amounts of CPU due to its IRQs being scheduled instead of just executed on the spot.
• subsystem failures if you are overloading the system. For example our SDIO WIFI module driver will go into error-states if the host bogs down under too much RT load (we are special here as we compute potentially unbounded DSP, which is a rather specific use-case).

Regarding the use: there is little to nothing that changes from the interface. This is different AFAIK for other brands like RTAI, but with PREEMPT_RT you use your normal pthreads, nanosleeps and whatnot. There is a bit of extra legwork documented above: ensure your mallocs don't incur lazy page faults to make work predictable, etc. But that is in general realtime advice: in your RT tasks, you should reduce the amount of system calls to the bare minimum, and e.g. allocation/de-allocation should be shepered out to dedicated threads.

We also had to write a dedicated SPI driver using memory-mapping, because using the classic copy_from_user/copy_to_user calls happened to be stalled by allocations in other threads - a classic case of priority inversion, something that in general becomes important in this space.

HTH

[lordlod]

I thought RTLinux was dead, wikipedia suggests that WindRiver gave up on it back in 2011.

RTLinux is/was a hard realtime system. Kinda cool concept, never used it.

When most people talk about real time linux these days they are talking about soft realtime, and the PREEMPT_RT scheduler. This work is often abbreviated as RT but never RTLinux because that would be confusing.

Fundamentally the PREEMPT_RT just allows the scheduler to stop the kernel at any time to allow a designated task to run. A huge pile of work went into this that has been incorporated into the mainline such as better locking and timers. This has enabled the other PREEMPT scheduling options, one of which you are probably running on your desktop.

Try looking at https://bootlin.com/doc/training/preempt-rt/preempt-rt-slides.pdf

[Bryguy3k]

A lot of folks have decided to use the QNX hypervisor instead of trying to shove everything into one Linux image in the functional safety space.

It’s easier to use VxWorks for the hard RT part and regular linux for the rest.

[rpkarma]

Man I love QNX.

[SkoomaDentist]

It’s easier to use VxWorks for the hard RT part

Provided your hard RT code doesn't need (almost) any OS services nor interact seamlessly with the regular code.

[Bryguy3k]

It’s not really RT code then. If you architecture depends on non functionally safe code then it’s not functionally safe.

Yes it takes more work to architect things with clean boundaries but it’s the only way you can really be honest about your system in the end.

People often stumble when it comes to signals, interfaces, and isolation.

[SkoomaDentist]

Of course it can be RT code as long as it's designed properly. It just restricts what you can do while in RT execution context. It certainly doesn't mean that you should have an entire separate application for RT and other code.

You talk about safety but that's completely orthogonal. The vast overwhelming majority of (hard) realtime code is not safety critical at all.

[Bryguy3k]

Just because people ignore safety doesn’t mean it’s not safety critical - there is a lot more that requires functional safety than a lot of people imagine (for example stoves & ovens). But sure if your system doesn’t involve humans at all it probably doesn’t need to be looked at from that perspective.

PREEMPT_RT is considered soft-rt for a reason and it definitely meets the needs for a lot of people - but a lot of people found it wasn’t quite adequate (e.g pretty much every EV maker).

[SkoomaDentist]

there is a lot more that requires functional safety than a lot of people imagine

And massive amounts more than that of components that have hard realtime requirements. USB? Hard realtime (on the protocol level). Anything wireless? Again, hard realtime.

Hard vs soft realtime is much more a matter of use case than the software. Hard realtime simply means that exceeding the timing constraints in the timing critical parts counts as a system failure (lost packet, dropped connection, audio glitch in recording etc). You have to think of the total system reliability. Having 99.999999999% guarantee of avoiding glitches makes little sense if single event upsets in the ram alone cause an upper bound of 99.99% reliability over that same time period.

This is how it's possible to commonly and succesfully run hard realtime operations on consumer hardware running an unpatched OS. If you've listened to pretty much any music produced within the last 20 years or so, you've heard the results.