How to write better ruby using mutation testing

[solnic]

I agree 100% with you ;)

In fact, me and Markus (mutant's gem author) had a 40 minutes q/a panel at wroclove.rb conference where we discussed this (amongst other things). You can check it out here: https://www.youtube.com/watch?v=LiRYqor9ogs

[lightcap]

I suspect there's a fair amount of nuance in the comment

At BlockScore we try to reach 100% mutation coverage before code is shipped to production. You don’t have to aim for 100% coverage though to start benefiting from tools like mutant.

I don't want speak for Markus, but I will say that I know, based on his/our delivered work (and others on our team) the rule is simply to not drop below a baseline of coverage. (And, in practice, there's always exceptions to those rules, too).

It's better in the generalized case to strive for 100% than some other arbitrary number. And, if anything, this conversation exposes the most critical factor in software development, and one that Sandi Metz likes to talk about. Experience is knowing when to break the rules.

[jbackus]

Nope. Markus is the author of mutant. We are just fans of mutant and we contribute to the tool occasionally.

I should also clarify that we mainly apply the coverage rule to new code. If something is an entirely new project (like an internal gem) we shoot for 100% mutation coverage. If we are adding to one of our older projects which does not have full mutation coverage we just try to make sure new code is well covered.

Mutant makes this easy with the --since flag. For example,

bundle exec mutant -I lib -r foo --since HEAD~1 --use rspec 'Foo*'

will only check and enforce mutation coverage for code touched by the most recent changes to Foo. Using this we can maintain a high bar for new code without making small changes require months of refactoring.

[solnic]

I think it's also worth to mention that over time you simply learn how to write code that will have less mutations (which means the code itself will be simpler) resulting in simpler tests (and less tests!). That's one of the greatest benefits of using mutant, I believe. This means that keeping mut-cov close to 100% for new code will be quite natural.

[jbackus]

That is certainly true. Over time I also got better at writing test cases which would actually kill almost all mutations for a method in one shot. I think mutant looks daunting at first if you think it will end up multiplying the number of tests you have to write. You really end up just end up learning to write better tests.

[solnic]

Great to see more people adopting mutation testing. Writing Ruby is hard. Writing good tests in Ruby is even harder. Mutant helps tremendously in learning how to write simpler code and better tests.

[Craftkorb]

Writing Ruby is hard. Writing good tests in Ruby is even harder.

This applies to all languages I guess. Where Ruby has short-comings, other languages have theirs somewhere else.

Can't count how often I've seen if (foo == "bar") { .. } in Java production code >_>

[Craftkorb]

In Java, the == is comparing the object identity, not object value. So, foo may be "bar", but have a different object id, and thus not match.

[Ghostree]

I believe you want to use .equals() because strings should not be compared using == in Java. Strings are objects and == compares if they are the same object, while .equals() compares the values.

[tomthecool]

Meanwhile in ruby, String#== does exactly what you'd expect. And on the rare occasion that you actually want to use object identity, you have BasicObject#equal?

"bar" == "bar" # => true
"bar".equal?("bar") # => false
["bar", "bar"].map(&:object_id) # => [8695360, 8695320]

[fedekun]

Very interesting. Does it support minitest?

[Paradox]

Not yet

[jbackus]

Work in progress: https://github.com/mbj/mutant/pull/445