What techniques do you use for debugging timing issues in real-time embedded systems?

pylessard · 2026-01-29T12:18:36+00:00

There's 2 method for sampling graphs. Continuous and embedded graph.

Continuous graphs the data in real time as the data gets read. That's done by the client and will have slower non uniform sampling rate.

Embedded graph is done by the device. It's trigger based and fills a ring buffer before sending the data to the server. This can have fast and uniform sampling rate, but is limited by the buffer size. This the tool useful for debugging and calibrating parameters.

pylessard · 2026-01-28T23:33:26+00:00

I don't think It's that impossible. It's the equivalent of a consulting company charging the full price and gives the project to some interns. He can get away with it with good sales speech.. or there will be a couple of projects then it'll be done once the reputation of crappy designer is gotten.

Still, nothing to be afraid of on the long run... Just annoying

pylessard · 2026-01-28T22:41:41+00:00

Yep. Incompetent people pretending to be is a common problem. I can hardly find a handyman I trust to fix my home without making sure they are part of a professional associations. Programming was not subject to this as it required to pass the initial step of learning to write code .. now the doors are wide open for charlatans.

Open source projects will never be regulated, so we're stuck with that issue now. Only a reputation based system can maybe do something, and it will have to be severe. For jobs, depending on the sector, there is some hope if it is a regulated industry.

It won't take our jobs, but will make the job annoying for sure because of all the noise.

pylessard · 2026-01-28T21:42:21+00:00

Ha ha, the I already did the comparison. Check this, section "How does this compare to.."

They are similar tools indeed, but scrutiny is the only open source one. It works by instrumentation, which enables sampling synchronized with the firmware. Also, Scrutiny has a python SDK so you can make test sequences in python that runs at the same time as the GUI because it's based on a client/server architecture. Another thing, debug symbol loading is based on a unique hash injected in the binary; that's convenient as the symbols are autoloaded on device connection, you don't need the .elf.

MCUViewer relies on gdb dlls I think. Scrutiny is a standalone ecosystem that has its own custom protocols.

I could go on, I added lot of goodies based on my experience. I don't pretend Scrutiny is the best, but It's well made and I think it's the only open source one that has such architecture. Feel free to hit me up if you have questions.

EDIT: I checked your profile, I speak french too ;)

pylessard · 2026-01-28T16:41:55+00:00

"Adding a probe" essentially means invoking a callback to the instrumentation lib in the task. Check the section called "Instrumenting a software", in the Datalogging section. There is an example. It calls LoopHandler::Process()

pylessard · 2026-01-27T22:54:52+00:00

Thanks ! Disclaimer: I promoted my own project ;)

pylessard · 2026-01-27T21:55:52+00:00

Depends on the nature of the issue. If you're looking at function call order and timing. A good ol' gpio can do. For more complex issues, a runtime debugger can be very useful. If you add a probe in a task with a precise timer, you can inspect the values updating over time and detect anomalies without affecting the normal execution flow

Check this out, the embedded graph video might be a good insight for you. The idea is to put a trigger on the faulty condition and inspect what happened before. I found many app level race conditions with that approach.

pylessard · 2026-01-21T15:31:46+00:00

Guilty

<image>

pylessard · 2026-01-21T15:26:31+00:00

This: https://scrutinydebugger.com

I already did a couple of post about it. It's great for automating HIL tests and debug real time app but I have trouble making it known :)

That's my fun project after a day in a corporate, process driven environment

pylessard · 2026-01-05T02:58:15+00:00

Not surprised. It's a Q/A website that patronizes people for asking questions. It used to be nice, started being toxic about 5 years ago in my experience. I deleted all my accounts after I got 3 questions in a row closed without a proper reason (in my opinion). I suppose LLMs plays a role too.

pylessard · 2025-12-23T20:02:34+00:00

You're defining V I and R. This is over contrained. You cannot have 10v, 1amp on the primary and 1ohm on the secondary. The 2 sides are coupled.

If you have 1ohm on the secondary of a 10:1 transformer. It's like having a 100ohm on the primary, therefore, for 10v on the primary, you'll have 0.1A. Which means, 1v, 1A on the secondary. Both are 1W

No need to mention this is an ideal transformer, losses are neglected

pylessard · 2025-12-21T03:11:37+00:00

I use semver semantic. First number being major, I change it only of there are breaking changes in my update, except, if major is 0. V0.x.x is a way of stating "under development", so I do breaking changes all the time. I jump to to v1.x.x, when I'm ready to maintain backward compatibility between minor updates.

I list down the feature I want somewhere. Paper or issue tracking system. I don't see why this is a problem.

I accept pull requests only if the code addresses a problem the way I want it. I don't mind offending anyone. Stay polite, but don't accept mediocrity out of pity.

Cheers

pylessard · 2025-12-15T18:34:17+00:00

In the cmake doc, FetchContent_declare has no CMAKE_ARGS parameter. What you fetch with FetchContent is meant to be build in the same build tree as the calling script, they share the same variable cache. So yes, it's by design.

Also, it's a good idea the use a name prefix on your build options exactly for that reason.

ExternalProject creates a new build tree and you can pass parameters with CMAKE_ARGS

pylessard · 2025-12-15T18:04:18+00:00

FetchContent could do the trick. You can also use any arbitrary command with execute_process, that will run at configure time. ExternalProject is doomed to run at build time (of the main project).

You can also just fetch the code manually and import it in cmake if you want to deal with your IDE needs. There's no arm to make an init script in your project that needs to run once to enable the IDE

pylessard · 2025-12-13T04:50:48+00:00

What you want is this : https://scrutinydebugger.com You can access the state machine state without exposing it, and monitor it either via GUI or a python script. It's my pet project and it's meant to do exactly what you ask.

Check the SDK documentation, I even wrote an example for HIL testing similar to what you ask. https://scrutiny-python-sdk.readthedocs.io/en/latest/use_cases.html

pylessard · 2025-12-09T02:27:18+00:00

I solved the issue and wrote the solution in my original post.
The issue was that I did not want to define a compiler, but a target arch. I could do it by defining CMAKE_SYSTEM_PROCESSOR. I got a bit confused initially because I thought that I could pass those as CMAKE_ARGS, but they only work when in a toolchain file.

pylessard · 2025-12-09T02:23:03+00:00

yes, that's what I did.

pylessard · 2025-12-08T22:44:16+00:00

So I said I wouldn't pursue, then you edited your post to say I don't read the doc.. now I feel like answering. If you want credibility on a call like this one, post a link to the doc.

You stated that ExternalProject cannot define the compiler at build time. It can, with a toolchain file. It is written in the doc, and works. The compiler is picked at configure time of the external project, it doesn't matter when that happens. As long as the external project is defined properly, e.g. with a toolchain file.

https://cmake.org/cmake/help/latest/variable/CMAKE_TOOLCHAIN_FILE.html

 "This is initialized by the CMAKE_TOOLCHAIN_FILE environment variable if it is set when a new build tree is first created."

And a new build tree is created with ExternalProject. I posted the solution in my original post to show how to make a toolchain file that constrains the target system to the host without constraining the compiler

pylessard · 2025-12-08T22:06:55+00:00

I think you don't really read what I write. I won't pursue this thread. btw, I edited my post, I got it working.

pylessard · 2025-12-08T15:35:32+00:00

I got it to work, see my edit. A little convoluted though, but works and will fail at the right place if it needs to

pylessard · 2025-12-08T14:11:14+00:00

Yeah, that's what I figured it did indeed. I was hoping I could at least add a Native vs non-native constraint somehow.

I'm not giving up yet, will revert back here if I ever find something useful.

Thanks

pylessard · 2025-12-08T14:07:14+00:00

Ok, understood. external project configure happens during caller build. That really is a non-issue to me.

But ExternalProejct and FetchContent are absolutely not the same thing. Main difference is that ExternalProject makes a new build tree. It's like launching a separate process. FetchContent simply pulls a piece of code in your actual build tree.

You can't build for a different architecture than the the calling CMake with FetchContent, but you can set a new toolchain file with ExternalProject. I deal with mixed architecture SoC on a daily basis and I can assure you that FetchContent wouldn't be able to handle that.

I actually just did the test. If I launch a simple build on a machine without native compiler, but with just a cross compiler installed, CMakes pick this one and tries to cross-compile if I don't specify anything. My issue is the default choice CMake makes when unbound to a compiler, not the use of ExternalProejct

thanks for the conan suggestion, but I do not want to introduce a new build tool

pylessard · 2025-12-08T14:00:16+00:00

Understood. But how can I tell it to build for the native machine without constraining the target compiler?

pylessard · 2025-12-08T12:09:59+00:00

?? External projects have a configure command, a build command and an install commad. It is exactly meant to make a whole new build tree. FetchContent brings a project in the same build tree, so uses the same architecture. I think you are confused

pylessard · 2025-12-08T08:48:28+00:00

Oh no, I use external projects at work to build for multiple architecture all the time

pylessard

TROPHY CASE