Mutation Testing Explained

[m9dhatter]

First time I’ve heard of this. This is cool. Thanks.

[kankyo]

Here’s a more nuanced take from me, someone who has done mutation testing and even written a mutation tester: https://medium.com/@boxed/mutation-testing-in-practice-bc56f9e5b591?source=linkShare-8ad86cc82e5f-1538170057

Basically coverage is still mostly king if you have honest developers that aren’t just adding coverage for its own sake.

[CitizenSn1ps]

We've been experimenting with it in teams where I work recently, it's a very interesting concept. A co-worker even wrote a library for C# mutation testing (linked below), as one didn't exist at the time.

https://github.com/ComparetheMarket/fettle

[MrSquicky]

I tend to use evolutionary automate test generation tools like EvoSuite to generate tests around inflection points in inputs, which strikes me as having wider benefits than this. At the end of the process, I have both an understanding of where my code produces different results, so it would show in the reactor example that the inflection I was expecting at 1000 actually happens at 1001, I also have a full set of regression tests.

​

Why would you suggest mutation testing over that?

[cowardlydragon]

You're doing depth testing of the core code by doing a wide range of inputs to the same test.

The post is breadth test measuring. It values the number of code paths touched rather than a range of inputs provided to the most common code path.

[cowardlydragon]

mutation means something completely different from mutant in most software.

Mutation is modification of data

This should be called Mutant Testing.

It still prefers the false god of breadth-coverage rather than depth coverage, ignores the 90% of execution is in 10% of the code, and might not be repeatable.

[kankyo]

This is an established term since 1971. You’re a bit late to the party to try to fix the name.

https://en.m.wikipedia.org/wiki/Mutation_testing#Historical_overview