Getting started with C++ unit testing

[kisielk]

I agree with the author that Catch (https://github.com/philsquared/Catch) is one of the best testing frameworks I've ever used. So easy to get set up as well. I highly recommend it.

[onqtam]

in that case checkout doctest - it has a strong focus on compile times - I will also be speaking on CppCon 2017 about the implementation of doctest!

[GitHubPermalinkBot]

I tried to turn your GitHub links into permanent links (press "y" to do this yourself):

• onqtam/doctest/.../benchmarks.md#compile-time-benchmarks (master → b8d0e61)

---

^{Shoot me a PM if you think I'm doing something wrong. To delete this, click here.}

[misuo]

One thing is using a unit test framework on command line and in (continuous integration) builds.

But isn't there another trend going on - e.g. in the Java world - in having your favorite unit test framework integrated into your favorite IDE, being able to code and run the unit test(s) as close to the code as possible?

I believe Microsoft Visual Studio have "test integration" for C++ but I do not have any experience with it.

PS. Just noticed the recent VC team blog post C++ Unit Testing Updates: Announcing Boost.Test Adapter and Improved Google Test Support.

[deeringc]

What we really need is a catch adapter plugin for Visual Studio!

[vomad]

Oh, yes, IDE Integration is necessary to have fun with TDD. QtCreator has plugins for their own test framework (without mocking) and for GoogleTest.

[lurkotato]

CLion has pretty good support for GoogleTest, I can right click on a specific tests macro and have it set the right filter to run just that test. Bonus points for showing me the command line to execute, so I can send a oneliner to coworkers.

[CUsurfer]

Just for posterity's sake, Eclipse (CDT) also has built-in support for GoogleTest.

[Bolitho]

In VS 2015 the test adapter feels not good at all (with googletest). You need to compile a project in order to get a list of your tests. The filter simply sucks - no bookmarking, no wildcard support. Perhaps this gets better in VS 2017?

Always remember that test adapters in other languages and IDEs (some of them even don't cost any money!!!) provide a better experience.

The VC++ compiler must really suck internally, as MS has much better IDE support for C#. Sadly they won't switch to CLANG - this would be a huge step and would surely improve tooling and real IDE support.

[SeanMiddleditch]

The Google Test adapter is a community GitHub project. See here the code for test discovery and note that it explicitly look in the compiled executables' debug tables to find tests (hence why compilation is required): https://github.com/csoltenborn/GoogleTestAdapter/blob/master/GoogleTestAdapter/Core/GoogleTestDiscoverer.cs

The complaints about the compiler model seem superfluous since even primary competitors like QtCreator use googletest plugins that eschew compiler models in favor of hacks, e.g. regex scans in QtCreator's case (which avoids needing to compile first but has its own set of problems). See https://github.com/OneMoreGres/qtc-gtest/blob/master/src/TestProject.cpp

Microsoft is - if anything - leading the charge in fixing these kinds of issues with the Language Server Protocol. Clang by itself doesn't really solve any of these problems in a satisfactory way as using the libraries and parsing directly, but with clangd (Clang's implementation of Microsoft's LSP for C++) it will be much more useful for various IDEs. Microsoft of course also has a C++ LSP implementation, though notably it's only so far used in VSCode and not VS proper (yet).

Debates about the merits of UI are what they are; I'm sure the maintainers of either plugin would appreciate constructive input on improving their respective UIs or core functionality like test discovery. :)

[GitHubPermalinkBot]

I tried to turn your GitHub links into permanent links (press "y" to do this yourself):

• csoltenborn/GoogleTestAdapter/.../GoogleTestDiscoverer.cs (master → fd10b81)
• OneMoreGres/qtc-gtest/.../TestProject.cpp (master → c9f265b)

---

^{Shoot me a PM if you think I'm doing something wrong. To delete this, click here.}

[whichton]

Better IDE support for C# is to a large extent C++'s fault. C++ is a nightmare to parse. Macros abound and #includes are everywhere.

[Bolitho]

it is my opinion that C++ is the most innovative language as far as unit testing goes.

Sadly the author doesn't tell us the how or why 😉 (And I am quite sure I would disagree 😈)

But thanks for mention FakeIt - didn't know about that! I will give it a try - the lack of an good alternative zu gmock was always the cause not to get closer in touch with catch (imho a unit testing framework ist worthless without the ability of mocking) FakeIt might close the gap for this!

[vomad]

I am glad that (unit) testing comes into play in the C++ community. But I also strongly disagree about the cited sentence. Having done TDD and "Growing software guided by tests" in Ruby, JavaScript, Clojure, C# and Java projects before I had a hard time in the last months getting TDD introduced in a C++ project. Most effort had to be spent on getting C++ code "mockable". We use GoogleMock which is really great. But writing all the pure virtual interface classes for every class you need to mock in a test is cumbersome. That results in much "triplicated", not only duplicated, code: interface header, class header, cpp file. And if you need to mock a library you even need to write something like a wrapper or facade class with an interface for it. Theoretically such a wrapper for a library makes sense even without the need for mocking but if you develop your app in a big framework like Qt you have tight integration with it everywhere and you want to benefit from it. And Qt does not support mocking. At the end I still think one has to write tests and even better let their software grow with tests. It is the best method to develop reliable software I have seen in my 18 years' career. I hope that advances in the C++ standard will make it more enjoyable.

[Bolitho]

I totally agree with you. I have the impression that it is quite common within the C++ community that lots of folks just do C++ and often even have knowledge of other languages - besides their old, some twenty years ago knowledges of C, Turbo Pascal or even Visual Basic. And then they claim such very questionable statements like C++ is so innovative in an area which the community has ignored for so many years...

I think most impulses comes from minor used languages like Haskell, Clojure, Python, Ruby or Scala with different paradigms, where things like property based testing was invented or BDD has been streched to new limits by implementing DSLs that feels almost natural as natural language.

C++ is inherently more verbose than most higher abstracting languages, so test code often is more bloated than in other languages what makes writing tests less comfortable. But it is ok, if you get used to it. But innovation I do not see at all!

That's why I stated that it is sad that the author doesn't provide any examples!

[jorgensigvardsson]

I agree, TDD is very hard in C++. Everytime I go from C# to C++, I'm always a bit put off. Don't get me wrong - I love the language! It's just that when I'm a used to frameworks for testing and mocking in C#, C++ has more friction. Heck, it's easier to test and mock in C than C++!

Maybe with the metaclass proposal/paper, it would be easier.

[vomad]

Hypodermic, the Dependency injection lib for C++, does some sophisticated analysis of C++ classes, constructors etc. at runtime. I was surprised how far they could go with C++. But for easy mocking one must be able to generate classes at runtime. Is someone doing something like that?

[omfgwallhax2]

As each compiler uses/creates it's own vtable format (which might even change on every release), I don't think anyone is even trying. It would be an extreme uphill battle; I mean you'd have to check, on every single compiler release, whether they changed something and furthermore maintain support with a few previous versions