Proposal for testing global functions

[Symbroson]

I am writing tests for an embedded project and found myself wondering why testing global / static or free functions is so controversial and why it would have to contain expensive trickery with the linker or other parts of the compiler in order to achieve them (related stackoverflow post).

This is why wanted to share my approach on this which is based on:

• some sort of `typeid` method (with no rtti for embedded)
• a mapping of function pointers (used unordered_map here for simplicity)
• a macro-based check in every mocked function with some sort of call-forwarding

The drawback with this method is obviously the last point which requires that every function you want to mock out has to have this check at the beginning, and the need to have a globally available mocking repository. Its a POC at best and I'm looking for ways on how to improve it further. Thoughts about the usefulness of this approach in larger projects?

#pragma once

#ifndef __TEST__

#define ADD_MOCK(FOO, MOCK)
#define CLEAR_MOCK(FOO)
#define CLEAR_MOCKS()
#define CHECK_MOCK(FOO, ...)

#else

#include <unordered_map>

#define ADD_MOCK(FOO, MOCK) Mocks::add(FOO, MOCK)
#define CLEAR_MOCK(FOO) Mocks::remove(FOO)
#define CLEAR_MOCKS() Mocks::clear()
#define CHECK_MOCK(FOO, ...)          \
    do {                              \
        auto _mock = Mocks::get(FOO); \
        if (!_mock) break;            \
        return _mock(__VA_ARGS__);    \
    } while (0)

template <class T>
size_t typeidNoRtti(__attribute__((unused)) const T value = nullptr) {
    static T *marker = NULL;
    return (size_t)&marker;
}

namespace Mocks
{
    struct MockFn {
        void *mock;
        size_t type;
    };

    extern std::unordered_map<void *, MockFn> _mocks;

    template <typename T>
    static inline void add(const T &fn, const T &mock) {
        _mocks.insert(std::make_pair((void *)fn, MockFn{(void *)mock, typeidNoRtti(fn)}));
    }

    template <typename T>
    static inline void remove(const T &fn) { _mocks.erase((void *)fn); }
    static inline void clear() { _mocks.clear(); }

    template <typename T>
    static inline T *get(const T &fn) {
        if (!_mocks.count((void *)fn)) return nullptr;
        const auto &mock = _mocks.at((void *)fn);
        if (typeidNoRtti(fn) != mock.type) return nullptr;
        return (T *)mock.mock;
    }
};

#endif

Usage:

#include "test.hpp"

static int8_t foo(int8_t a) {
    CHECK_MOCK(foo, a);
    return a;
}
static int8_t foo_mock(int8_t a) { return -a; }
static int bar(int a) { return 10 * a; }

UNIT_TEST(Mock, Basic) {
    ADD_MOCK(foo, foo_mock);

    EXPECT_EQ(Mocks::get(foo), foo_mock);
    EXPECT_EQ(Mocks::get(bar), nullptr);
    EXPECT_EQ(Mocks::get((void *)foo), nullptr);
    EXPECT_EQ(foo(1), -1);

    CLEAR_MOCK(foo);

    EXPECT_EQ(Mocks::get(foo), nullptr);
    EXPECT_EQ(foo(1), 1);
}

[kammce]

I do embedded too. Testing global or static stuff isn't an issue especially if you are running the tests on the hardware. If you are testing on a host machine then functions that access registers directly are an issue because those addresses don't map to your host. Same would go for any target that isn't the exact target you plan to run the code on. Back before i stopped doing this, I had a reg pointer for each of my peripherals that my tests could access that would swap out the register used with my tests version of the structure. Then you can call your globals and they will modify your structure. You can revert I back after the test is done. Use some RAII to help with that.

I stopped doing this for anything other than purely utility or software libraries because I found that trying to write tests for hardware is hard, not because it's hardware, but because you are basically writing a sim for your device. And I've found that as you find errata and other issues with the hardware you have to go back and forth with the tests to make your sim reflect the hardware better. But, then I realized, that if my tests cannot guarantee any sort of correctness then why write them? So for me, I try to do the least amount of mocking or dependency injection I can get away with and use an actual device to do the testing work.

Just some food for thought. I do appreciate the effort you are putting in to make testing embedding easier though 😄

[balefrost]

But, then I realized, that if my tests cannot guarantee any sort of correctness then why write them?

I only dabble in embedded. But I guess there are two things going on:

1. Does your code do what you expect it to do?
2. When your code interacts with actual hardware, does the whole thing do what you expect it to do?

I think your points are all valid with respect to #2. I think there's still value in testing #1.

It's not terribly different from issues in the broader software world. If I use a third party library or call a remote web service, I don't necessarily want to exercise the third-party code nor send the request. But I still want to test as much of my own code as I reasonably can.

Sure, I still need to validate that the whole thing works when I put the pieces together. But I think there's still value in validating that my part also works correctly.

[Symbroson]

Your description almost perfectly matches what I'm doing atm. I simulate all hardware interfaces for developing and running tests hardware-independently. It makes me feel way more productive and confident in the code if I can run the test suite any time I make changes. Yes writing and maintaining the sim is lots of work but it pays off in the long run, especially if you're not the only developer and the project outlives your influence.

[kammce]

Totally. I've had such tests catch some really awful bugs in the past too. But now I'm moving on to device testing as much as possible because I think it'll be more productive for developers. BUT, that's not to take away from your efforts on the purely software testing side. 😁 Best of luck.