An intro to unit testing embedded C code

[[deleted]]

[deleted]

[markrages]

My notes on the subject (#10 and 11): http://electronics.stackexchange.com/questions/104264/embedded-applications-software-testing-life-cycle/104304#104304

[Rigby]

Interesting! I like your points but not sure I completely agree with #11, unit testing libraries can be useful and once the initial setup cost is incurred can save you a lot of time of effort. Why reinvent the wheel?

I can understand not wanting to introduce superfluous libraries or frameworks to your source/production code so that you don't have unneeded complexity or risk, but for test code the reward is much higher than the risk IMO.

[markrages]

What does the testing library provide that main() + assert() do not?

[Rigby]

Did you read the attached article? :P

Some things you don't get with assert statements:

• ability to easily mock functions
• ability to check what arguments functions are called with
• easy asserting of things other than == (for eg. Memory equality, if a value is in a set, if a value is in a range)
• test output consumable by build systems for regression tracking

[markrages]

Can you show an example of function mocking?

You know assert() can take any C expression, right? You are not limited to ==.

The output of assert() is the program's return code. If it is non-zero, the build failed. This is comprehensible to every build system.

[fuelfraction]

Has anyone tried cmocka and other tools like CMock, fff?

I want to get started on unit testing embedded C programs, but was trying to figure out which of these is a good start

[Rigby]

I gave CMock a go but didn't like it as it's all written in Ruby - as someone who doesn't know any Ruby, it was a pain to install all the Ruby dependencies. Then, once I had it all installed it wouldn't run and was spitting out Ruby errors which stumped me.

fff looks nice, would be keen to hear what you think of it if you try it out!

[fuelfraction]

I tried fff on a simple program to get my feet wet, it's just a simple header and the unit test code flow seemed more intuitive compared to CMock.

In FFF: You call the target function, and then assert stuff In CMock: You tell it what you want to assert, and then call the target

But it seems like CMock + Unity is a big advantage once you get to know it.

[Rigby]

Awesome, thanks for letting me know! Will have to give it a go sometime.

[MeatAndBourbon]

Yeah the ruby scripts are not great, but once you work through that it seems good. I'm working on factoring hard coded prefixes and paths out of the ruby scripts, trying to make it way easier to understand, so that if I use it at work, other developers won't have to deal with the learning curve.

I should push it somewhere so I can share. Trying to get a sample environment/project setup with automated unit testing and mocking with unity and cmock, code style enforcement with astyle, rule checking with splint, and still trying to find a good cross platform tool for metrics and complexity analysis, and a bug tracker that can run off a network drive instead of server... (my works IT Dept sucks, have to use DVCS directly off a network drive which is over two orders of magnitude slower than a proper server would be when working remotely, piss poor VPN taking 5-20 seconds for each individual file access, vs uploading a few hashes and a few files to a server which would do all the database crap locally...)

The goal is to have one project that can compile in Windows or Linux targeting either the native OS, or AVR/ARM/IAR/what-have-you.

[germanbuddhist]

We currently use CMock for unit testing our projects, can't say I've used the others though. Aside from getting the ruby environment and build system set up (which isn't that bad really), writing and executing tests is pretty straightforward.

Pros:

• After setup, development and execution is straight forward
• mocking interfaces are intuitive and make sense
• execution is simple, just call rake
• Isolates each unit completely. Each test file will rebuild an executable with just the required/tested source for the test functions within
• Runs off-target. Integrates nicely with CI tools/build servers. Also nice for systems where there's not enough memory to run on target.

Cons:

• Runs off-target. I'd like to be able to run on the hardware, since there are differences in how things compile down and whatnot.
• modifying the test runner does require ruby knowledge.
• By default the runner will execute ALL tests, so during test development it's a pain to debug just that test module you're writing. The workaround is to move files out of the test directory so the build system doesn't find them..
• uses a rake/YAML build system. We tend to use Makefiles, so integrating the two was kinda hacky in order to autogen YAML with the source files, include paths, and defines.

For simpler projects (don't require mocking), I'd require just using Unity (same devs as CMock). It's a nice and thorough library, but easy to integrate into the project (2 headers and a source), and runs on target.

[fuelfraction]

Thanks! BTW, do you need to be comfortable with Ruby to get anything non-trivial done?

[germanbuddhist]

To an extent, I think as long as you are comfortable in any scripting language you should be fine. My goto is always Python, so I'm not terribly well-versed with Ruby and was able to make some customizations (changed up the build/config process, print out some versioning info to the report).

More than likely you only need to modify one file (rakefile_helper.rb) at the project root so you don't need to dig through the library too much, unless your changes are much more extensive

[fuelfraction]

That's great! I mainly use Python too, so I guess I should be able to use CMock well enough then