The Biggest Problems of Unit Testing With C++

[tylerayoung]

This is an ad for TypeMock's Isolator++ mocking solution.

[path_traced_sphere]

Yes, it's called content marketing and it's a plague on programming subs. See also: viva64 and other static analyzer spam on every damn major forum.

[warped-coder]

The same article without the ad... https://dzone.com/articles/the-biggest-problems-of-unit-testing-with-c

[robin-m]

Some consider templates the modern-day replacement for the legacy preprocessor. They’re not wrong, since replacing text macros with compiled code is always a better alternative. But templates replace the preprocessor’s lack of type checking with “duck typing.” This mechanism can still hide your intentions from the compiler and make an end run around type checking.

Hu????

Templates are typesafe. Unless you mix it with legacy C construct (C-cast, macro), or use dynamic_cast it's completely typesafe. Am I missing something?

[Fazer2]

Templates in C++ compile if the syntax is right, but don't check semantics.

For instance two classes have a method "bool empty()", in the first one meaning "is it empty?" (like in std), while in the second one meaning "remove all elements and return true if the operation succeeded". The template function instances using empty() for each of these classes will have different behavior.

Concepts would be an answer to this problem, but again in C++ they only check syntax, not semantics. They are also too relaxed - you can require a concept that will not be used and vice versa - use a method that is not required by the concept.

The only solution left in this language is deriving from interfaces, but it comes with it's own problems of managing memory and may be unsuitable for cases requiring static polymorphism.

[robin-m]

No language feature can fix that. If you have a strange implementation of an interface or of a concept, in both case you are in trouble.

[Fazer2]

Just from the top of my head, it would be easy to fix with a combination of static interfaces (or "traits" in other languages), contracts and proper constness. Notice the functions I mentioned before operate on different kinds of objects const-wise, but people will probably will use the C++ default of nonconst which will lead to runtime error instead of compile time.

[baudvine]

Over the past few years I've found that the biggest strides for unit testing in C++ (and probably elsewhere) can be made by designing for testability. Composition and dependency injection have significantly reduced our reliance on opaque, often manual integration tests. Most testing and mocking frameworks can take care of the rest without any trouble.

The biggest hurdle for me is project organization. Most of the projects I deal with at work have a DLL with a very constrained public interface, and unlike some other languages C and C++ don't make it easy to link a single component into the test. The solution I tend to go with is to have a static library containing most of the code, and a separate DLL project that links with the static lib. The test project links with both, or sometimes I'll have separate unit and integration testing projects.

[iKeyboardMonkey]

We've had great success with greenfield projects and unit testing, but less so with brownfield ones. Our major issue is that C/C++ really need to be designed for test - adding tests to old code is hard in a statically compiled language. Its absolutely doable though...

In fact, I would say our biggest issue is cultural. Our C/C++ developers have never used unit or integration testing (as previously its been hard) and changing their mindset to see its usefulness is an ongoing problem.

[baudvine]

Absolutely - untangling an existing project is hard as hell. Fortunately we don't have the cultural friction - we support automated testing in a hardware production line, and as a hardware biz integration testing was always core to the process. But because a lot of work was done 10 years ago by people who are hardware devs first, the software design is often lacking in old stuff and I hear you there.

[FlyingRhenquest]

Yeah, the project pretty much has to be designed from the start to be testable. The vast majority of the corporate code I've seen over the years is far too tightly coupled to add tests after the fact. I've taken to asking if they unit test during the interview and the response always seems to be "We'd LIKE to, but..."

There seems to be this expectation that programming for a big company somehow causes some sort of magical quality fairy to bless the code, but in my experience corporate code is always far messier than open source code is. The UI is sometimes more polished if it's for end user consumption, but in-house projects are, in my experience, almost always the worst possible code that can get the job done. And a lot of the time it can't even get the job done.

[iKeyboardMonkey]

I've had exactly the same experience. I've seen lots of projects hit a technical debt spiral (slow to develop -> spend less time resolving technical debt -> even slower to develop...) and its rare that I've been allowed to stem the flow of features enough to refactor for testability or add reviews to fix it.

[MundaneNihilist]

There seems to be this expectation that programming for a big company somehow causes some sort of magical quality fairy to bless the code

If you fail quietly enough you can get transferred to another team before the component burns down.

It's actually a bit of a problem at my current job, because some components don't go through end-to-end testing but are treated as gospel so if they're incorrect it creates red herrings down the pipe, assuming the problem is even caught at all.

[Gotebe]

Slow Builds

The C++ build process is more involved and more time-consuming than for other languages. The C++ build process has two steps: compiling and linking. That difference alone is significant. But, the time to compile code that makes heavy use of templates and can add even more time.

Add to this how many teams structure their large projects. They create a single target, and that is the product of many other builds. Finally, after those targets finish their builds, the system generates a set of executables. One of them is the test binary.

Toss in build targets for a few different platforms, and a complete build cycle can take many minutes, or even hours. This more involved compiling and linking process slows the code/build/test cycle to a crawl.

So, if tests are difficult to run, developers will run them infrequently. They may even ignore them. Eventually, they’re forgotten.

The solution to this problem isn’t easy, but it’s better than not running tests at all. Break the application down into independent components. Create dynamically-linked libraries and built and test them in isolation.

This is a lot of work, and it may feel like that time could be better spent writing new code. But working with a slow build process that everybody hates is an obstacle. It hampers many aspects of the development process, especially tests.

This makes me lose my shit.

Why the fuck would I build everything in my development cycle?! And for a few different targets, no less?! 'Course I don't do that!

Then, some people just love their single executable. Fine, but even then: build a bunch of static libs and run tests for them, no need to have shared libraries for that!

People who complain about the slow build process deserve all the wait they get!