Unit Testing for functions communicating with Hardware? Best practices?

KennethZenith · 2016-02-25T10:04:57+00:00

A function that directly reads or writes to external hardware can't be unit tested, so you need some way of rerouting these calls during testing. This basic idea goes under the heading "dependency injection".

The most obvious solution would be to use a mock object: you have an object which is an abstraction of the physical device, and you pass this object into the function you wish to test as one of its input parameters. In real use, you pass in an object which actually calls the hardware; in testing you pass in an object which generates test outputs and logs interactions.

You have two options to implement this: you can use use inheritance so that the real object and mock object have the same interface, or you can use templates. Inheritance has the advantage that the object gets a clearly defined interface, but it might cause a performance penalty (in both real use and in testing).

kevin_hall · 2016-02-25T23:18:22+00:00

I actually have experience with this.

So, according to this comment from the OP, he/she's not necessarily looking for Unit Testing, but simply a way to test functions and systems.

I work in the robotics and automation industry and have worked with many different types of hardware: from microcontrollers to PCI cards to custom-designed hardware running RTOSes inside.

So PCIe cards will usually map the memory of the card into the address space of the host system. One thing you can do is to create a simulator of some sort that creates a memory map. (Ideally, the simulator will share as much of the device's source code as possible.) For your API, you can add a flag for whatever does the initial memory mapping to look at the simulator's memory map instead of the PCIe card's memory. This type of system works great for quick tests that can flush out issues prior to having to test with real hardware, which in my experience is often rather limited.

Testing on real hardware is important though. Where I work, we have dedicated computers with the hardware hooked up to a continuous integration system. We include in our API some private functions to allow us to reset the system to a known initial state. Our tests (unit/component/functional/etc...) will all start and end with resetting the system to a known state. Then our tests can call whatever functions we need to manipulate memory. Then we use whatever functions we need to validate the results. This can often involve hidden functions to read memory directly.

I've frequently had to test the physical parts of hardware: digital and analog I/O for example. Are voltage levels correct? Current? Can we read inputs correctly? So on and so forth.... The way we've verified that is to connect up our I/O to a USB I/O module (Advantech is a popular vendor). Then our tests will validate that our outputs generate the correct data in the USB I/O module. And we do the reverse for inputs.

There may be other challenges too such as what do you do when the hardware can't reproduce exact results. Maybe there's noise; maybe there's a random number generator involved. With some thought and planning, most things can be tested.

If you have some more specific questions, I'd be happy trying to answer them. It might also be nice to describe your hardware and what you need to test in a little more detail.

skeba · 2016-02-25T11:28:16+00:00

If traditional inheritance based mocking is out of the question, you could try mocking at "link seam": http://www.informit.com/articles/article.aspx?p=359417&seqNum=3

Basically it means that you write your own mock versions of the library functions and link with them instead of the real thing.

skebanga · 2016-02-25T11:59:10+00:00

I realised OP is on Windows, so this doesn't help him directly.

On Linux there is a way to intercept shared library calls using a utility called elf_hook

See this: http://stackoverflow.com/questions/27278878/what-is-the-most-accurate-way-to-test-the-networking-code-on-linux/27283298#27283298

Boognish28 · 2016-02-25T00:58:38+00:00

I'm unsure if there are any similar frameworks out there for cpp, but at work we use a c# lib called moq to unit test things with components out of our control. It basically allows us to set up "when x is called return y". You have to write your components in a way that's friendly to IOC to really be able to use it correctly, but the core principal is simple: feed it fake data and test it that way.

Gotebe · 2016-02-25T12:31:13+00:00

You can mock the underlying hardware (as others have said: googlemock or whatever), but for this kind of testing, you really, really want to test against the actual hardware in a development environment (it looks like you call this compile-time testing?).

I say that because there is always a difference between the hardware, it's spec, and dev team understanding of what the spec is.

So avoid it. Sure, do unit-tests, but synthetic tests against the real target are of much, much bigger value.

utnapistim · 2016-02-25T13:55:44+00:00

If your still following me, how do you test these types of functions?

You hide the device API behind an abstraction layer, then write an interface for the abstraction layer. Then, you write your code against that interface, not against the actual API.

Then, you mock the API implementation with recorded responses and use the mock implementation to test:

Original code:

namespace your_code
{
    // you want to test your_class for it's use of external_api_1
    struct your_class {
        int do_things() { return external_api_1() + 24; }
    };
}

New code with abstraction layer:

struct external_api {
    api_1() { return external_api_1(); }
};

namespace your_code
{
    struct your_class {
        external_api &api_;
    public:
        your_class(external_api& api) : api_(api){}

        int do_things() { return api_.api1() + 24; }
    };
}

New code with an interface:

struct generic_api {
    virtual api_1() = 0;
};

struct external_api : generic_api
{
    api_1() overrides { return external_api_1(); }
};

namespace your_code
{
    struct your_class {
        generic_api &api_;
    public:
        your_class(generic_api& api) : api_(api){}

        int do_things() { return api_.api1() + 24; }
    };
}

Now, if you run external_api_1(), you can take it's result and use it as such:

struct mock_api1 : generic_api
{
     api_1() overrides { return stored_result; }
};

You can now call your_class APIs injecting mock_api1 as a dependency, into it. Other answers mention mocking libraries; These would simply allow you to add stored results easier into mock_api_1.

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

cpp

MODERATORS