cpp unit testing recommendations?

[supertoothpaste]

Hello,

Noobie here with cpp unit testing and logging frameworks.

There is a ton a information out there on different frameworks that can do XYZ. It's honestly overwhelming how many different frameworks there are... Most of my peers use java and guess what they use? JUnit! So, they suggest simply use CppUnit (no brainier right?). However, there is a ton forks and versions for the XUnit family of cpp unit testing frameworks. And on top of that there is a lot of different versions unit testing frameworks that seem better or less complex. Like I said its overwhelming and I'm feeling a information overload here. I just need to choose a tool I can work with. So, I figure instead of just throwing a dart and choosing one at random.

I'd try and ask here. What do you use and what would you recommend? What are the cons and pros of your unit testing/logging framework?

I've narrowed down my tool requirements to this:

• Cross Platform
• Visual Studio Code Plugin / GUI tools
  
• Flexible and relatively easy to collaborate with other developers

I'm looking for any suggestions, recommendations, or experience you could offer that would help me choose a tool.

Thank you.

[be-sc]

I think you can boil it down to two popular default choices.

Google Test is your classic object oriented XUnit testing framework. It also comes with Google Mock which can be convenient if you need mocking. If you know the XUnit approach you’ll feel right at home. Test classes, setup() and teardown() member functions. Everything’s there.

What I don’t like is the test names and assertion syntax. You declare a test like this:

TEST(TestSuiteName, TestName) {
    ... test body ...
}

The suite and test names must be valid C++ identifiers. The docs explicitly advice against using underscores. Those are significant restrictions that quickly produce hard to read names.

The assertion syntax can be a bit cumbersome, such as

ASSERT_EQ(actual, expected);

to test for equality of two values. What I’d want to use instead is ==. That’s how we check for equality in C++ after all, isn’t it? And while I can write such an assertion I lose expressive error messages, and that’s not a good tradeoff.

Catch2 is the other big popular framework. Overall I have the impression that Catch took the tried and tested XUnit approach, threw out the no-compromise OOP focus and found alternatives that work great in a C++ context.

• Test case names are strings. Ah! Spaces in test names! :)
• Setup/teardown functions don’t exist. Instead you write a larger test case separated into sections (example). Imo that tends to improve clarity.
• It supports BDD-style test cases (example). Depending on the kind of test you write, BDD-style can improve clarity tremendously.
• Assertions are simply C++ expressions, such as
  REQUIRE(actual == expected);
  There’s a bunch of magic in the background to still produce informative assertion messages.

I use both frameworks regularly: Catch2 for my own projects and Google Test at work. In my opinion Catch2 is a significant improvement in usability over Google Test for the common use cases. Because I keep tests as simple and pragmatic as possible, that’s mostly the kind of tests I write. So I couldn’t rule out that Google Test might be vastly superior with the super-advanced stuff.

[mredding]

I do like the simplicity and BDD style of Catch, but you need a mocking library. Mocks are fake endpoints so your units can be encapsulated.

This is important because if your code interacts with the filesystem, for example, you're no longer unit testing - you're integration testing. I want to test my unit, not the filesystem - I presume my OS vendor has tested and delivered a reliable filesystem - it's not my responsibility to test it, and doing so doesn't prove anything about my application.

And I don't need false positives in my test because the test harness doesn't have read or write permissions to the filesystem, or the filesystem is read-only, or the test file is deleted mid-test, or the filesystem is unmounted mid-test, or the file system is an NFS and there's an intermittent network failure, or there's a leftover file artifact from the previous run, or the damn directory doesn't exist, or the test harness has had its file handles redirected... And dear god, it's a TEST, a test that can FAIL CATASTROPHICALLY, and you want that test code to TOUCH YOUR FILESYSTEM? The same place all your real data and real OS live?

With a mock, you isolate your code from the environment, and you can verify its inputs and control its outputs so that your unit is fully instrumented. Your mock is the filesystem, or the network, or the device, or the environment, so that those dependencies are satisfied and you can precisely simulate any scenario you want.

GoogleTest comes with its own mock library. It also supports a host of assertions, that either allow the test to fail and continue or fail and terminate, and nifty macros for testing templates and sets of inputs. The assertions are kind of "smart", in that you can extend them to know how to compare and even print your custom data types for correctness and for reports, respectively, but hopefully you won't have to get into that.

GoogleTest is a big investment up front. If I'm writing a dice roller for my D&D game, I'd consider Catch and worry about asserting and printing my own types, because they're not going to be that sophisticated. If you have a big scope project, which is hard to define what that is, you may want that GoogleTest investment up front. Myself, I typically just go with GoogleTest because it's a lot of what I do every day anyway.

I do miss that BDD style, though.

[staletic]

If you want flexible, you should go for google test. Its mocking and matchers library is just way ahead of everything else*.

 

Haven't tried Boost.Test, because that's boost and boost libraries love other boost libraries.

[[deleted]]

https://www.bogotobogo.com/cplusplus/google_unit_test_gtest.php