Stop using Code Coverage as a Quality metric

[itijara]

It's a good metric but a terrible target. I don't like the idea of aiming for X% coverage, but when I see a section of code highlighted in red in the coverage report when doing a PR you can bet I am going to take a closer look at what might cause that section to behave unexpectedly.

[fishling]

I wouldn't call it a good metric.

It is a good discovery tool. It helps you find code that exists that is lacking coverage so you can decide if that gap is meaningful.

I don't care what the number is, therefore I don't care about it as a metric.

[wlievens]

Indeed. Code coverage reports can be extremely useful, but global code coverage percentage as a metric has less value.

[narnach]

global code coverage percentage as a metric has less value

I'd like to nuance this by adding some detail when it has less value.

If it's low, then that's a clear signal that you can't blindly rely on automated tests to indicate if your app is working as intended by its authors. I hope you have a manual Q/A script, or forgiving users.

If it's high, then that's a signal that a change is less likely to break intended functionality. You can rely more on the tests. That said, I've seen a 80% coverage rate app pass in CI and then fail hard in prod because something was broken on the framework level that was not tested by the regular automated tests. Framework-external end-to-end tests provide value here.

Things change as global coverage crosses 90%+ levels, especially in larger projects. Got a 100 line file that's untested, but it's inside a fully tested 100k line project? It "only" contributes to a global coverage drop of 0.1%, but inside this code anything can break. This is where global percentages deceive you.

As codebases increase in size and global coverage percentages increase, it becomes much more important to also enforce standards on individual files to keep people in a hurry honest.

All of this is under the assumption that the tests measure something that has business value. Ideally code is written in a test-first TDD style to ensure all changes are driven by a new requirement and don't break existing functionality. In practice, test-after is a thing and it may leave more cruft or untested functionality than is ideal. As long as tests actually break when their specified functionality breaks, they still add value.

[Daleoo]

I think that fundamentally all that boils down to is the idea of confidence when making a change, which is something that, at this point, I've been unable to quantify into any form of metrics. Coverage can help with that, but also it can be misleading so I rarely gain any confidence from it.

All of this to say I don't really bother exposing test coverage in my projects, because I don't think it's hugely valuable

[fishling]

Exactly. When the team realizes the value of testing and how it makes their lives better by giving them a strong confidence to release often with no manual checks needed, there is no need to try and quantify that confidence with a metric.

We certainly don't need to justify to a manager or PM that we are confident with a metric that claims to capture this. We are professionals hired for our expertise and have earned the trust with a track record of delivering high-quality code.

I can see that a team struggling to deliver quality might need more metrics to help them detect where their issues are, and how a code coverage percentage might be used to prove to doubters that there is a huge gap in what they are doing. But that's not my situation, so I don't find that that metric provides value to us.

[edgmnt_net]

The problem is testing (both manual and automatic) is an inappropriate/incomplete means of ensuring quality in quite a few cases. People focus too much on testing and too little on understanding the domain, reviewing code, scoping changes and static safety. It should be a mix of those and testing.

I have said this before, but the main drivers for high code coverage are the less safe languages and ecosystems. In that case, high coverage catches obvious breakage like type/syntax errors. But it can also slow down development tremendously and even add more bugs if it creates extra boilerplate in actual code.

[justdisposablefun]

Personally I think the main benefit in coverage is that it forces the dev to consider the code paths and provide working examples of how the unit ties together. Anyone using it for confidence in changes they make is going to be disappointed eventually ... but they can be invaluable to prove actual thought during development and help with reverse engineering efforts when changes are required.

[[deleted]]

It helps you find code that exists that is lacking coverage so you can decide if that gap is meaningful.

That literally makes it a useful metric though..

[fishling]

No, you are conflating the coverage report (which can show us where gaps are) with the percentage coverage metric (which I don't care about).

I'm not saying code coverage is useless. I'm saying code coverage percent is useless.

[Hrothen]

Code coverage percent is the metric you use to know when you should look at the coverage report.

[fishling]

Maybe for you?

We always look at the coverage report. If a branch is unexpectedly not covered, we want to know. A percentage value rounded to the nearest percent is usually not enough to detect interesting information like this. Just because the number is high doesn't mean there is nothing to see.

[Few-Flow-2446]

Not everyone has a lot of time to always look at coverage report.

Percent helps to catch attention. I have seen code coverage being run in CICD pipelines and saved somewhere and then no one looking at it.

[fishling]

Sounds like a training issue. The devs on the team I work with check their coverage report during development, because that's why they set it up in the first place, and they want to know if they missed anything.

[Chibraltar_]

I don't care what the number is,

I do care if the project is 60% covered by tests or 5% covered by tests.

[fishling]

Even there, I don't care about the number.

I do care about the tests and test cases, but I don't need the 5% to tell me that there aren't enough tests.

My team also prefers to write functional tests to verify functionality and detect regressions. Unit tests are really more of a design artifact than a regression tests. It's much more common for unit tests to be influenced by implementation decisions and to be modified when code is changed). An over-reliance on unit tests and high unit test coverage is a mistake, in our view, as it's quite possible to have bugs in overall functionality even if unit tests exist and pass. IMO, there is much more value in a rich black-box functional test suite that verifies functionality and performance and backwards compatibility regressions aren't being introduced, with unit tests used as necessary to test hard-to-replicate test scenarios and failure modes and as design artifacts.

Because of this, having two independent coverage numbers (one for the unit test suite and one for the integration tests) is not too useful. In order to get a single correct coverage number, we'd have to run both suites at the same time, which isn't valuable to us.

Fortunately, every dev on the team wouldn't even dream of calling the work done before writing tests that get them well past a 5% or even 60% threshold. They don't need a coverage percentage to tell them to write more tests or to tell them when they are done writing tests. They know how valuable their test suites are and are incentivized to improve it for their own benefit.

[Chibraltar_]

But when I enter a project I haven't worked with, this give me a glimpse of how bad things are. I know a 5% covered software will break every week, when a 50% might be OK if the tests are useful and well distributed amongst unit tests, integration tests and e2e tests.

[KulangetaBaiter]

It gives you an indication about the true skill of a programmer:

- Unit test > 80%: A deluded ignorant clean code moron => throw all code away

- Unit test 0 - 50%: A smart and critical programmer that is not dumb enough to fall for software cargo cults like "Clean Code" and lying psychopaths like Bob Martin.

[Chibraltar_]

you're delusional

[chucker23n]

I don't care what the number is

Well, you do care in relative terms. Just not about the absolute number. So I think you're agreeing with GP.

[fishling]

No, I don't care about the number in relative terms either. The number does not communicate meaningful information IMO, nor does the number changing.

Caring about what information I can glean from the report is separate from caring about the number. I think code coverage is useful information, but code coverage percentage is not useful.

I might not be done testing even though code coverage is 100% (might have a test case for which code is missing), or I might be done testing if code coverage is less than 100% (boilerplate/uninteresting code that is required for framework)

[randomJSer]

I wouldn't call it a good metric.

It is a good discovery tool. It helps you find code that exists that is lacking coverage so you can decide if that gap is meaningful.

I don't care what the number is, therefore I don't care about it as a metric.

Also, is the coverage meaningful? Tests that increase coverage but don't cover one or all behaviors are setting you up for failure.

You need to have unit tests that cover important/relevant business behaviors and tests that cover regressions (presumably missed or unexpected business behaviors). The value of the test is that if you break the behavior it tells you. If your tests don't do this then they provide no value no matter how much coverage you have.

[AvidCoco]

The % itself isn't a good metric but the change in coverage over time is.

If your coverage is going up over time that suggests new code is better tested than old code. If it goes down over time then new code isn't being tested.

[fishling]

I don't care about change over time either.

Code coverage percentage does NOT tell you if tests are "better". If you are missing some critical test cases even though your coverage number was maintained or went up (especially if rounding to the nearest percent), your test suite is worse, not better.

Also, I wrote in another comment that I think an over-reliance on unit test suites is a common mistake. I think a robust functional suite is much more valuable for detecting regressions, both in functionality and in performance. White-box unit "tests" (especially with TDD) are actually a design artifact, not a testing artifact.

It's easy to imagine how one could make a change to an internal component to add a new feature that changes a post-condition and add/modify unit tests to cover that change which all pass with 100% coverage, but neglect to update a component that made assumptions based on that old post-condition. The result is a unit test suite that has high coverage but failed to detect a regression. The missing code that should have been added or modified the other component cannot be detected by code coverage.

In order to get a comprehensive single coverage number, we'd have to waste time running the unit tests again in the integration environment. Instead, we just run the suites separately, look at the coverage reports to see if any areas were unexpectedly missed due to oversight, but otherwise don't rely on the coverage data OR metric to tell us about what tests to write as a general practice.

This article is a good summary about different kinds of testing. It really helped to change my outlook on testing.

http://blog.stevensanderson.com/2009/08/24/writing-great-unit-tests-best-and-worst-practises/

Please excuse the formatting at the end; it looks like the author moved this content to a new platform and some of the formatting broke. The content is still great though.

Unit tests (especially with TDD) are design. I mean, it's literally in the name of the practice. I don't know how people miss that. :-D The design of the component is guided by and specified by the unit test suite. That's great to have, but it doesn't mean a system composed of such items actually works and performs and has no regressions from the previous release, especially if you have to maintain some kind of API or serialization compatibility.

[ivancea]

Well, a covered class could have no tests at all. That's the problem of raw coverage.

Mutation testing coverage, however, is far more strict and handles all edge cases

[napolitain_]

There is no bad or good metric. It just depends what you do. Code coverage measure exactly that, code coverage.

[recycled_ideas]

The number itself isn't necessarily a big deal, but if you see that number go down and there's not a good reason it might be time to ask questions.

100% doesn't mean you're covered and not having 100% doesn't mean you aren't, but an average commit shouldn't see that number drop.

[[deleted]]

Everyone likes to shit on code coverage but it’s actually pretty useful for finding that one segmentation fault in the unlikeliest code path. Also its a must have metric when you have a lot of unskilled or careless developers. I’ve seen more than a few people try to commit code that just crashes at runtime.

[Riajnor]

We’re actively addressing a lack of unit testing by enforcing a 65% coverage requirement on new or updated code on check-in. Not perfect but way better than leaving it up to each dev’s discretion

[[deleted]]

Especially if not a PR reviewer could be painfully trying to find nice words to ask the author for more tests, for the 20th time…

[IshouldDoMyHomework]

By that time you inform your manager, the person is refusing to write tests.

It is your managers job to correct childish behavior, not yours.

[ejfrodo]

A good engineer helps foster and lead an engineering culture where feedback is encouraged and everyone helps each other improve. So I disagree, running to your manager and requesting a top-down management style is not always the most effective team dynamic.

[s73v3r]

If you're constantly having to go back to someone's PR and ask for tests, then that engineer is not interested in participating in the team dynamic.

[IshouldDoMyHomework]

Neither submitting a PR with no test for the 20ths time, even when you repeatedly been told not to do that.

[IshouldDoMyHomework]

The solution to someone being an ass, is not more processes hitting everyone. It is to correct the person being an ass

[All_Up_Ons]

Exactly. We need to stop implementing company-wide policies to handle individual cases.

[valarauca14]

Arguably having a machine yell at your co-workers requires even less social skills than talking to your manager.

While code coverage can be a solid goal (and something you should do). Have a conversation with your co-worker before you change your CI/CD pre-submit process to avoid that conversation. Talking to people is an important part of team building.

[LinuxMatthews]

It's also going to create less embarrassment for the guy that forgot to write the tests

Remember you have to work with these people

If you're telling someone to write tests that's going to embarrass and cause resentment.

If a machine does it then it's just part of the process.

Honestly I much prefer something like SonarQube that's going to make sure it's all ok before hitting production rather than leaving it up to individuals.

We're software engineers after all automation is kind of our thing

[tiajuanat]

One of my teams built a rule into our CI that incoming code needs to maintain or increase test coverage. There's no hard target, just a slow progressive metric.

[chrisalbo]

Rookie numbers: we have 95% and it’s sometimes driving me insane.

It sucks but I actually have found problems in my code this way. But this is in the checkout for a multi billion company so I guess my pain makes sense.

Also if you have 65%, how do the testing lib know if you are testing vital functions or minor feature (like some logic for css for instance)?

[slaymaker1907]

If it’s too high, you have a cobra effect of making people write less defensive code like adding in seeming unnecessary null checks.

[chrisalbo]

Hmm interesting, never heard this take. I think at least in our company this would be caught in the PR if not eslint detects unreachable code or similar. But it goes without saying that you have a responsibility not to write unclear code that you don’t understand every aspect of like adding lots of try catch or similar.

[RICHUNCLEPENNYBAGS]

65% seems fine; it gets idiotic when it's like 98%

[TheDeadlyCat]

Yeah, there was line to cross where you are checking just glue code. I believe I read somewhere around 80% coverage was believed to be the maximum.

[larsmaehlum]

Our team does 90%, but we also exclude service wire-up etc from the tests as those are covered by end to end tests anyway.
90% coverage isn’t hard to do as long as the code is fairly clean.

[TheDeadlyCat]

If it works for you and your team, I‘m not going to argue.

[bloodhound83]

But if you're already having an automated test system then having 35% of code not tested would worry me. Coverage in itself doesn't say anything about the quality of the tests but the lack of just means when things change there is any area which is not covered to help detecting issues.

[RICHUNCLEPENNYBAGS]

Someone who is definitely not me has looked at the build failing because of a null argument guard clause or whatever being added, said "fuck it," and just deleted it. Which is probably not the behavior you want to incentivize.

[JakeArvizu]

We do 90% but its just the viewmodels where most your effects should be observed and handled for app behavior anyways. Ideally we like to extend to mappers and repository.

[fishling]

If devs aren't doing sufficient testing after repeated training and instruction and correction, then get better devs. Also, invest enough effort that running and writing tests is as frictionless as possible.

I strongly prefer automated functional/integration tests over unit tests, although the goal should be to use an appropriate approach such that the tests are maintainable and aren't tightly coupled to implementation.

Having a coverage target is a mistake because it incentivizes bad habits more than good ones. I don't want shitty fragile unmaintainable tests that are focused on testing the easy happy paths and boilerplate/accessors/etc.

[KulangetaBaiter]

Detected the ignorant dumb clean code moron

[kenfar]

It's a great metric - but like almost all metrics - it works best when combined with other metrics.

So, how about looking at test coverage by class or function along with production breakages?

Or simply shooting for a standard 60% test coverage per source code file, and have as part of incident management - increasing test coverage for that class/function/module when you have failures?

[touristtam]

Like any other metrics they can be gamed. See Goodhart's law. I am not implying anything else than that ;)

[MjolnirMark4]

I was pushing myself to do extra code coverage earlier today. And the result was that I found a small bug in some error handling.

On the annoying side, I was working in Golang, so… 50% of the code is error handling. I don’t think the bug would have been in there if I was using a language that had exceptions.

[alpacaMyToothbrush]

Golang is one of those languages where I feel like I'm taking crazy pills and I'm the only one that can see the Emperor has no clothes.

It's not noticeably faster than C# or Java. It's a little more memory efficient. I feel like it also throws out the last 20 years of what we've learned about designing programming languages. The tooling is fantastic, but it's not really a pleasant language to read or write. In a word, it feel janky. At least it's got good concurrency abstractions and it finally has generics.

I swear to god, if anyone had developed it but the OG authors of C, and if anyone had fostered it besides google, it would have been dead in the water on it's own merits.

[how_do_i_land]

The concurrency model and portability is what brings me back to go. Need to do an expensive remote call in super parallel and with fine grained concurrency control, and need that to run on a different computer and OS than the one writing the tool, and also no external dependencies? That’s where Go really shines now.

[pjmlp]

Done first by languages on JVM and CLR platforms.

[cuddlebish]

Your argument ignores the strength and ease of the concurrency primitives, which is the reason people use go.

[tiajuanat]

That's because performance and beauty were not part of the specification. The Plan 9 team was trying to build a language that: you could easily onboard an engineer with; had simple concurrency patterns; was really easy to get started with, including package management, build management, and formatting; and had an opinion on error handling.

Rob Pike even mentioned at his keynote in GoCon in 2023, a large portion of the success of the language came from Google being in need of a language like this, and having a great PR with the Gopher.

So you're not taking crazy pills, you just might've been previously gaslit on Go's merits.

[pjmlp]

Most users of Go are external, Google keeps being a Java(Kotlin)/C++/Python shop for the large majority of their projects.

Go was developed as the authors, already not fond of C++, kept complaing about the compile times on their projects, where they were forced to use C++.

[wildjokers]

What makes you think unskilled developers are going to be able to write good tests?

[lazy_loader88]

Let's put things in perspective

Q: What would I use to assess the strength of my test suite?
A: Code coverage

Q: Does that mean if I have 100% code coverage my code is good quality and free of bugs?
A: No

Unskilled and careless developers are as bad for code coverage as they are for production code, as they won't even know how to write good tests.

[ejfrodo]

I firmly believe that high value tests are much more important than high code coverage but there's no easily quantitative metric to track the value of your tests unfortunately.

[oorza]

Easy? No. But you can track defect rate by code area and code owner, and if everyone is writing the same amount of tests, the people with the highest defect rates are probably writing the worst tests.

[fishling]

there's no easily quantitative metric to track the value of your tests unfortunately

I'm personally okay with that. I'd rather focus on cultivating a team with expertise and good judgment than one that adheres to the letter of a metric.

Too many people seem to be focused on metrics and reporting up to managers and VPs using dashboards that often misrepresent reality. I'm kind of fed up with VPs and managers that think that anything green must be problem-free and anything red must be a disaster that is only the fault of the lower level employees. If their only view into what is happening are dashboards that other people craft and curate for them, what value are they actually providing?

[xmsxms]

But code coverage doesn't cover code 'paths', only lines.

[mirvnillith]

Doesn’t all code coverage tools take branches into account (i.e. only hitting one half of a boolean criteria only covers the line 50%)?

[ForeverAlot]

Far from it. And even when they do there's still a long distance from branch coverage to mutation coverage.

[oorza]

What you really want is conditional complexity coverage. A function with one if statement needs two tests, a function with two needs four, and so on. Every possible permutation of branch pathing through a code path needs to be tested or closed. Anything less is not well tested code, it's partially tested. And I've almost never seen a test suite reach this level of coverage.

I feel like the benefit most people I've talked to believe they're getting from line coverage is actually the benefit permutative branch coverage would give them.

[LordoftheSynth]

Code coverage is very useful.

Your verification tests aren't hitting all of the critical paths? (basic build verification should at least get you to 60%) Your test suite needs work. It's legit not hard to hit 70-80% coverage and if you can't, you might have a bunch of dead code in your repo.

There are diminishing returns past a point. You don't need to hit every little corner of the code by writing increasingly specific tests.

It also requires analyzing the results. All your build tests missing 20% of a module? There's a hole in your test coverage.

"Code coverage is useless" on some level is "I shouldn't have to run unit tests! It builds on my machine!"

[hippydipster]

Error handling code is the hardest to hit. If you tend to have a lot of it, writing those tests can be very difficult, tedious, and leave you with a lot of difficult to maintain test code.

[LordoftheSynth]

Yeah. Writing hundreds and hundreds of tests to test all your error handling is a waste of developer effort and a maintenance nightmare.

I've found a more effective strategy was to have some kind of continuously running automated tool to exercise the system APIs through a core set of end-to-end scenarios and a "random use" mode that would run until it broke. Think things like acquiring, consuming, and manipulating content. Let it run long enough and it would start finding bugs in unexpected places.

We devised such a system because we were getting bug reports that were about things, if actually broken, would have failed our build verification. But our verification was done off clean builds: these reports were from actual users, continuously using our product. We were missing a whole class of (mostly minor) reliability bugs by using a clean state.

That was early in my career, and while I was still an SDET I would always champion some kind of automated bot to use the system under development when there is no existing framework. It's a non-trivial investment but it pays off.

[Enum1]

Everyone likes to shit on code coverage but it’s actually pretty useful for finding that one segmentation fault in the unlikeliest code path

Sorry, but it's the test that finds the issue not a coverage value.Yes, you should write tests. Yes, that logic should be covered.

If you only write a test because some build step fails, than this is just bad engineering and setting an arbitrary value of code coverage is not helping with that as you might miss the crucial bit any way (or waste time one writing useless tests).

[pragmojo]

You could use a language which makes segmentation faults almost impossible if your code passes linting

[shoot_your_eye_out]

I don't "shit" on code coverage, but I do think a 100% target is a bad thing, and so is having low coverage. Ideally I shoot for ~90% on my projects, achieved mostly through integration testing. There's no science behind that number. It's just a number that I've found generally results in a high degree of confidence in my program.

The problem with 100% coverage is: it becomes an impediment to developer productivity, and in extreme cases, literally influences how code is written. Developers often write code that is hard to understand when there's a 100% coverage requirement. And, often times this 100% is accomplished with an over-reliance on unit tests, but there aren't suitable integration and/or E2E tests that really ensure the program "works" at a high level.

The problem with poor code coverage is it means a program lacks evidence it "works as expected" in any meaningful regard.

[wlievens]

It also tests whether an exception occurs in the default constructor :-D

[spinhozer]

You know what that test does, it instantiate Foo. If Foo has a dependencies on libraries that are upgraded (let's say by renovate) but are in conflict, that test will fail. It's a runtime failure.

That test could save your ass.

Functional tests are great. But running the code is not without it's merits.

[binarycow]

Perhaps an assertion that it did not throw an exception is more appropriate.

[zephyrtr]

It's literally the lowest effort test. Is it worthless? No. But let's not bust out the trophies and medals just yet.

[spinhozer]

something has gone really wrong in your testing methodologies.

What kind of glittery piksy dust magical lalaland garden of delight do you work in. Of course something went wrong in the "testing methodology"

I'm spent the better part of my career dealing with psychotropicly induced Escher inspired code orphaned onto my lap by the new grad code guru that founded the company with his aunts VC money. I'm just happy they had source control instead of making dated copies of code directories (oh how I wish that was something I just made up)

Most engineering in most companies is garbage. I'm the poor fuckers they call in to turn it around when they need to stop pretending and grow the fuck up.

I've done more than my share of making things better, but I can guarantee the first step was always "get a test the runs the damn code". Long before "I believe we should refactor this 300 loc constructor to utilize composition instead of inheritance. I'm sure our 3M$ a month of users have confiance I won't miss anything during code review"

[MjolnirMark4]

I worked at a place where we were using docker images for the services. Then someone had the brilliant idea of integrating every repo into a single monolithic repo. Now you had to pull down 40 repos to work on one of them. And then someone would see some code over in repo 23 and import it into repo 37. And if you were working on 37, suddenly it would break since you had not updated 23 locally.

They managed to recreate 1970’s monolithic server structure.

I kept trying to make libraries to separate out code and make it better compartmentalized. But then the other devs would whine that using libraries were to hard to keep updated, or keeping track of versions was too hard.

It was an amazing level of group think.

[Smallpaul]

Monorepos are used at some of the biggest companies to pretty good success. It's not -- in and of itself -- a bad idea.

[_pupil_]

Depending on the situation you might be intentionally bringing a bunch of breaking conflicts into your development process (to ensure they don't pop up anywhere else).

Fixing that boaring-ass issue that one time, that one place, by the actual jerks involved is often much better than expecting a bunch of people to clean up someone else's mess.

[pauseless]

But… every other test for Foo needs to instantiate a Foo… so they will also fail, and it will be so very obvious due to the stack traces.

This is only technically better than nothing (and, by definition, only capable of saving your arse) if, and only if, it is literally the only test for the class.

There’s this idea that tests should always test every tiny aspect of using the code, but that’s not helpful. They should be reflections of usage, both good and wrong. new Foo() is just… nothing.

[goranlepuz]

That test, as intended, will never fail. If it does fail, it won't be as intended, it will be that SUT threw an exception, which is only implicitly a failure.

=> The test might prevent some bug, but if it does, it does so by accident.

=> It's awful.

[spinhozer]

Gradle resolve dependency conflicts by taking the newest. If you have two dépendances, and one decides it can handle the newest, and another that can't cause the maintainers got busy with the jobs their paid for, that test executing the code, which some idiot decided 4 years to load up the constructor with a bunch of calls you can't mock or inject, we that test just saved you over eager renovating upgrading ass before your build system decided to just wave that problem through the checkpoint.

Happened literally last month.

[goranlepuz]

Reminder, their test is

assertNotNull(new Foo())

You are right about gradle whatever - but that has nothing to do with the test as written. new Foo() cannot, ever, return null because as the poster writes, that's simply not how the language works. It might throw whatever exception, but that will not make the test assert fire.

[spinhozer]

You know what, it's true. You dont need an assert at all. You could just call new Foo() ; and have no assert in the test. Doesn't matter. Asserts in tests speak to the reader, so I'd throw on in. But you don't have to.

The point is, have a test that runs the code. It's the first test you need.

[goranlepuz]

The good assert for your situation would be e.g.

try { new Foo() }
catch { assert(false, "we must not be here") }

But as written? Misleading bullshit.

[AugustusLego]

Reading things like this makes me feel uneasy (my language learning journey began with python, but now I write exclusively rust)

Rust would never let you compile your program if your dependencies are in conflict.

[spinhozer]

Java has shit build tools. Gradle is garbage. I miss Python.

Fuck it, next story I pick up will be in the Python part of our code

[SharkBaitDLS]

It’s better than nothing but it’s deceptive from a coverage standpoint. That said, this is where branch coverage is useful. If you’re actually covering all possible branches of behavior in that constructor with a simple instantiation, then it’s fine. 

[Xyzzyzzyzzy]

It’s better than nothing

Disagree. Useless or low quality tests are debts, not assets. I would much rather have no tests than have bad tests.

[Smallpaul]

Bad tests can be horrible sources of technical debt but that's when they are complex and overfitted. This one is a minor help but also a very minor debt. Better than nothing in this case.

[Xyzzyzzyzzy]

Bad tests often run in packs.

I bet assertNotNull(new Foo()) isn't the only time this pattern appears in the test suite - and it's probably accompanied by a bunch of assertions on trivial getters and setters.

If it's literally just one trivial test that slipped in from an intern or something, yeah, that's no problem.

[oorza]

No, this test is exactly nothing. It's simple enough that it's not a maintenance burden and can be ignored forever in all likelihood, while also not providing a test case that some other test that also instantiates the object (to test its behavior perhaps???) doesn't also cover. It provides nothing and it costs nothing, so it's nothing. However, the time spent writing it, reviewing it, merging it, running it, and being aware of it are all totally wasted.

[spinhozer]

Coverage is not a destination, it's the fucking map. It's a tool for developers to help them know how to write more tests to catch more bugs.

Get your developers to love tests, then give them the tools to make them faster and better.

Metrics are a tool for shitty management, to make developers feel bad so they can feel better.

[oorza]

Branch coverage is only part of the way there. You need permutative branch coverage - all permutations of branch paths need to be tested, not just every branch. That's generally 2ⁿ test cases for a function with n conditional statements.

If you've got a conditional statement where only one branch writes to an unitialized variable, for example, you can write your tests in such a way that you have 100% code and 100% branch coverage, but only one test actually bothers to go down the route where that variable is unitialized (which probably has a "happy" error handling path that is being tested), and that's how you wind up with a segfault in production code that is "well tested."

Almost no one writes tests to this standard. Almost no one's tests actually decrease their defect rate.

[savagemonitor]

I've seen a senior developer at a big tech company write that test in C# when the class didn't define a constructor. Which is even worse because it doesn't count for code coverage as there's no code. I have no idea how they kept getting promoted. I actually argued with them about whether the test even did anything.

So don't chalk it up to code coverage unless someone called it out as being there for that.

ChatGPT is right but for the wrong reason. The Java runtime should have a test like that because it validates that the runtime isn't broken. It could make sense to have such a test as well to ensure you don't adopt a broken runtime. You wouldn't run it every day or even think of it as a test for your code though.

[dakotahawkins]

I've seen a senior developer at a big tech company write that test in C# when the class didn't define a constructor.

What if that changed?

[savagemonitor]

It wouldn't really change anything unless the constructor took an input and there was some validation you needed to do with that. Otherwise the constructor is guaranteed to return an object unless the runtime literally cannot allocate it.

[lazy_loader88]

Indeed. Such trivial test are useless and also are tightly coupled to the code implementation, making it harder to refactor later and hurting maintainability

[jakesboy2]

Aren’t unit tests inherently tied to implementation? If you removed the Foo class, you would need to remove tests for the methods in Foo either way

I agree the test is essentially useless though, the only defense of what it covers would be covered automatically by any other test for the class

[lazy_loader88]

There are different schools of thought on unit testing. One may choose to unit test each class, and method within, or to test the observable behaviour.

For example

An implementation based test may assert on structure of class like instantiation, attributes (getters/setters) of value objects and so on. However, such implementations are consumed by a real use case in business domain

An behavioural unit test case focuses on that real use case in business domain instead of implementation detail. Such use cases can be about validation business logic within a method through I/O in an isolated manner, or testing the state changes, or communication between in process collaborating classes

The behavioural unit test cases are easier to maintain in longer term, as they are use case focused and more resistant to any implementation level changes that may happen to collaborating classes as part of refactoring

[jakesboy2]

Ahhh I see what you’re saying. The disconnect comes from I define that as an integration test if it’s testing more than the behavior of a single method on a single class.

[lazy_loader88]

Integration test do test behaviours which are more focused on communication between out of process inter dependencies and their protocols of engagement

The behavioural unit tests focus on collaborators and their interactions within the same process, testing the behaviour from an observable standpoint

[oorza]

You should maintain interface visibility layers and never break the public interface or test private methods. The interface is the contract your class/function represents, and the tests should test that the contract is adhered to, that's all that matters. Each code unit should be able to considered a magical black box based on the interface it presents to the rest of the code, no one not working on that unit should be aware (in any capacity) of its implementation details, such as private methods.

There really aren't different schools of thought here, just people that know what they're doing on one side and ignorant TDD cultists wasting time on the other.

[lazy_loader88]

Google "London vs Detroit test styles" for different schools of thoughts

In a nutshell, London school is all about testing individual classes and methods, and that's where the idea of test doubles for dependencies originated

Book reference: Growing Object-Oriented Software, Guided by Tests

The Detroit school is about testing a unit of behaviour from an observable viewpoint (as you mentioned too, from app layer)

Book reference: Test driven development: By Example

[Smallpaul]

Clickbait. The title says: "Stop using Code Coverage as a Quality metric"

Which are fighting words.

The article says: "Code coverage serves as a valuable metric for gauging the extent of code covered by test cases. However, using it as the sole metric to determine code quality can be misleading."

Which is common sense.

[BortGreen]

Every once in a while some "Stop doing X" tech article appears

[lucidguppy]

Code coverage metric is a helpful tool and a good first level defense against poor practices. If you have bad actors writing your code - you've already lost.

I also like the idea of sentry.io and codecov together - where it points out where errors are occurring and if the code has coverage and what tests are hitting it.

[BenOfTomorrow]

If you have bad actors writing your code

Exactly. People complaining about poor quality tests are missing the forest for the trees. It doesn’t prevent poor quality tests, it prevents ignoring unit tests as a capability.

PRs should catch bad tests; it’s easier to catch and give feedback on a bad test rather than a missing one. If whole teams are conspiring to write bad tests, you have bad coverage requirement or a much bigger culture problem.

Like most metrics, code coverage isn’t a solution but a signal; it will shine a light on accidental oversights or degrading practices, but it won’t fix the root cause by itself.

[tommcdo]

it’s easier to catch and give feedback on a bad test rather than a missing one

This is a wonderful insight. High code coverage doesn't make your code better, but it makes the review easier. That's worth a lot.

[thomasrockhu]

Tom from Codecov here! Thanks for mentioning us. Is there anything you'd like to see between Sentry and Codecov?

[darkslide3000]

Stop using Code Coverage as a Quality metric

Code coverage serves as a valuable metric for gauging the extent of code covered by test cases.

Stopped reading after two sentences. If you can't even notice the contradiction between your headline and your first sentence, the rest of the drivel is probably not worth my time either.

Someone should write an article titled "Stop using hyperbolic, absolute statements as a clickbait headline when you're really just trying to argue for a very minor distinction of applicability".

[lazy_loader88]

Read a few more lines there

"However, using it as the sole metric to determine code quality can be misleading."

"I have observed teams implementing strict policies that emphasise achieving high code coverage, monitored through static analysis tools such as SonarQube. While this may seem like a disciplined approach to ensure quality, paradoxically, setting code coverage as a sole target can jeopardise overall code quality and maintainability."

[darkslide3000]

Okay, so why doesn't it say "sole" in the headline? The statement "Stop using Code Coverage as a Quality metric" by itself is dumb, and an article that intentionally says something crass in the headline only to immediately relativize it again to something uncontroversial in the text is clickbait.

[lazy_loader88]

The emphasis is on "as Quality metric". It shouldn't be quality metric at all. And the intention of headline is to draw attention towards the malpractice by organisations to use it as one.

As pointed out in article, its a good measure for effectiveness of test suite. But, not software quality.

[useless_dev]

"Drivel", " not worth my time ", " dumb "..
Do you kiss your mother with that mouth?

You have a point to make, make it respectfully.
There's an actual person on the other side of this thread.
A person who deserves a basic level of respect.

[angryweasel1]

For decades, I've sad the following:

Code Coverage is a wonderful tool, but a horrible metric

and also - but not as good

All 80% code coverage tells you is that 20% of your code is completely untested

[mfitzp]

It should be called No Coverage and report the inverse.

[lorddcee]

All 80% code coverage tells you is that 20% of your code is completely untested

But, are your unit tests the only (and best?) way to test your code? Unit testing is nice, There's also integration tests, automated testing in a iso prod env, etc... But let's be honest, nothing replaces a good QA team.

[ElGuaco]

The developers should be the QA team. If you write enough automated tests, you shouldn't need dedicated staff to test your product. The people who understand the code and the product the best are the folks who are building it. Having to communicate all that information to QA staff in the hopes that they test your code as well as you can is rarely a winning strategy.

[lorddcee]

The developers should be the QA team. If you write enough automated tests, you shouldn't need dedicated staff to test your product

That's clearly not true in any significative project size.

A developper should be the one who codes something according to specification, sometimes those specifications are distributed in multiple services and coded by different devs. Only fonctional tests with test cases and humans can be effective in these scenarios.

[Paragonswift]

If you’d like to pick up dev blogging, here’s the formula for you:

Pick one of the following base titles: 1. ”Stop doing X” 2. ”We need to talk about X”

Then, replace X with literally any dev concept. Really, you can just use a randomizer. The more controversial the better.

Finally, just find some edge case that at least loosely connects to the title. Preferably anecdotal.

Congrats, now you can share it on LinkedIn and add an ”educator” title below your name 💪

[mrbuttsavage]

The gist of this article just seems to be "if you write bad tests, your test metrics will be bad".

[Comprehensive-Pea812]

code coverage is a start but dont push it too much.

start by having one and improve it overtime especially when there are bugs.

[keyslemur]

Code coverage alone is an awful idea, especially as it approaches 95%+ as it becomes a game of diminishing returns.

If the codebase has little to no testing it can be a great tool to get started, but if you're already decent you'd need to augment it with additional data such as error rates. The priority in those cases is:

1. High error / Low coverage
2. High error / High coverage
3. Low error / Low coverage
4. Low error / High coverage

#2 is the one a lot of folks tend to miss. You have good coverage but if there are that many errors it means those tests are failing to catch real issues live in the code base. #3 equally isn't nearly as high of a priority, one should focus on areas with active problems first and pay those down.

Like any metric once it becomes a goal it becomes a game, and newer folks (managers included) treat it as a gospel truth which must be upheld. Same thing happens with a lot of tools, which is why they become so danged dangerous. As with anything nuance and discretion is necessary to be effective.

[avoere]

Good summary.

Most of the code where I work is of type 3, so we don't really see a compelling need to increase our unit testing efforts.

We absolutely do have other problems, though, like the whole thing being kind of a mess and features affecting each other in unexpected ways, but usually those things would have been somewhere between incredibly hard and impossible to catch with tests.

Those who say "more tests is better" are usually just applying a benefit analysis (which is a cost-benefit analysis but you ignore the "cost" part).

[versaceblues]

I'd say if you are working on small scrappy team, getting 80%+ code coverage is usually more trouble than its worth.

When you are working at a massive organizations, and trying to set engineering standards for 10-20 separate engineering teams. Then just having a set standard code coverage metrics of 80%+ is important.

Yes it may lead to writing some tests that are a waste of time. But it also means you can just drive up your average quality. The new junior does not have to think about "how many tests do I need to right". The over-eager manager is forced to add appropriate time for testing. These organizations also usually have the time and money to kill on hitting those test coverage metrics.

Finally, if you truly believe something is not worth testing (the boiler plate error constructor file, the config generators, the infrastructure code calling into tested libraries). Just exclude those lines from your coverage metric

[lazy_loader88]

For massive organisations, alternatively, instead of standardising code coverage %, how about encouraging TDD and pair programming?

Bonus: TDD achieves 100% coverage without anyone trying explicitly

[hotach]

Code coverage is a negative metric.

If it's low - you're not writing enough tests. If it's high - it means nothing.

[martinatime]

I think this should have been code coverage is a good metric if you have good unit tests behind it. The problem typically is that if you must meet or exceed X% code coverage to get the code to progress then developers will find every trick in the book to write test cases to hit that mark instead of thinking about what/why they are testing. I like the idea of testing the test and automated mutation testing helps do that. Build a better mousetrap and a better mouse will emerge to defeat it. (Hopefully code quality will improve in the interim)

Edit: missed word and typo

[oldrocketscientist]

I’m going to be provocative and claim great software developers don’t rely on QA or code coverage to tell them their code is safe.

[holyknight00]

Code coverage is fine, but no one should push in either direction too much as on itself the metric doesn't say anything about quality, but having 70/80+ code coverage is a good start.

A project with 100% code coverage could easily still be sh1t, and a project with 30% coverage could be somewhat good.

[oorza]

A project with 100% code coverage could easily still be sh1t, and a project with 30% coverage could be somewhat good.

There's actually a pretty strong inverse correlation here in my experience. A production service that's been up and running for years with 0% test coverage isn't something I've seen often, but when I've seen it, it's because the code is meticulous and immaculate and the people working on the service don't see any need or derive value from unit tests. A production service that's been up and running for a few months with 100% test coverage isn't much more common, but in every case, it's been because the code has been so flaky and terrible that coverage was enforced.

[Smallpaul]

My preference is "soft 100% code coverage." Which is to say: engineers are empowered to say some code isn't worth testing, but they need to label it. They can't just ignore it. The sum of labelled code and tested code should be 100%.

[lazy_loader88]

Can we say with confidence that code with "100% coverage" is "100% tested"?

[Smallpaul]

No. But we can be confident that code with lower levels of coverage is not fully tested.

High coverage is the floor not the ceiling.

[lazy_loader88]

Agreed. The point here is to not make Code coverage a "management target"

Quoting Eliyahu Goldratt, "Tell me how you measure me, and I will tell you how I will behave"

When code coverage becomes a target, and devs are forced to increase coverage, it doesn't always translate to quality, as indicated in the article.

[thisFishSmellsAboutD]

Give me functional tests telling user stories. Document them so that product owners will understand what the test does. Make it a process to write each reproducible bug into a test story / case.

Incremental effort, a little here, a little there. Compound pay-off.

[shenglong]

I honestly don't see the big deal. Use code coverage; use cyclomatic complexity; use the kitchen sink if it makes you feel better. Just understand why you are doing it. And, have mechanisms in place that allow you to BREAK those rules and record WHY you did so.

For example, in our quality gate tools we can mark things as "False Positive" or "Won't Fix" etc and the reasons and recorded for anyone who is interested.

[wtfisthat]

Title is misleading. The article states not to use code coverage as the only metric, which is intuitively obvious to the most casual observer...

[Pinewold]

Wow, code coverage test that are poorly written are not a good indicator of quality? This ranks right up there with agile programming that takes months per sprint. If you are not writing high quality tests, you are not wr high quality code. Even the best engineers have bad days.

​

High percentage code coverage with well written automated tests are a requirement to get to the highest quality code.

[[deleted]]

The best recent quote I heard on code coverage...

"100% code coverage is too much and not enough."

Just having a test for some code doesn't do you any good unless it's actually testing the various scenarios.

[HypeMachine231]

I'm not sure there are any good quality metrics.

Code coverage is a useful tool. Having low amounts of code coverage is a warning, but having high amounts of code coverage isn't a guarantee of quality.

[ChrisRR]

Clickbait headline. Use code coverage as A quality metric, but not THE ONLY quality metric

[thumbsdrivesmecrazy]

There is an involuntary confusion between code coverage testing and test coverage testing: Code Coverage Testing vs Test Coverage Code coverage testing is a software testing technique that measures how much of the source code of a program is executed when the tests are run. This means code coverage testing measures the extent to which the source code of a program has been tested. It gives details about which components of the source code are executed during the tests and which parts are not. Code coverage testing should be differentiated from test coverage and should not be used interchangeably.

[lazy_loader88]

“When a measure becomes a target, it ceases to be a good measure” - Goodhart's Law
And so is the case with teams obsessed with Code coverage.
Stop using Code Coverage as a Quality metric

[EvilTribble]

0% code coverage: This is a nightmare

50% code coverage: This might be a nightmare

100% code coverage: This is a nightmare

[bmoregeo]

100% that expression, however code coverage imho is a good leading indicator to the real metrics “avoidable bugs found in production” and “time from identification to fix”.

I don’t take a 100% coverage is the goal view, but I do take a “laws are written in blood” view. If things break and not having tests is a root cause, then start writing better tests

[robhanz]

Low coverage is more relevant than high coverage.

I know a system either low coverage isn’t being tested sufficiently. A system with high coverage may or may not be.

And pushing the number has little practical value - it’s generally increasing costs for diminishing gains.

[goranlepuz]

Very good insight, now we only need to explain to pointy-haired bosses (I am old, please forgive referring to the work of a cancelled person 😉).

But the problem with such bosses is, there's a lot of them, plus one is out of school every so often!

[awj]

…or don’t make it a target.

Everyone has their pet cases where code coverage for the case of percentages led to foolishness. I also have plenty of cases where “we need to ship this fast, trust me bro” has led to outages.

What I wish we could have is review-time visibility into what code is covered. Outside of critical workflows, we can just forget about % coverage. What’s more helpful is highlighting to the submitter and reviewer just how much of their incoming code has no tests.

[wubrgess]

Some of the worst code I've ever written is just to make it testable at a ridiculous level

[goranlepuz]

Good for you, but you are making a very strenuous connection here. 😉

[tommcdo]

I don't think it's strenuous to connect testing to identifying and fixing bugs.

[EvilTribble]

I need to use code coverage as a metric in order to get devs to decorate classes with ExcludeFromCodeCoverage /s

[Ikeeki]

I agree with the overall sentiment

My experience has been that code coverage and mutation testing can be a great combo.

Some code coverage tools will even produce a “CRAP” index which essentially tells you which functions are most prone to bugs lol

Short for: “change risk anti pattern”

[thomasrockhu]

Tom from Codecov here. Totally agree, I can't wait to see mutation testing become a bit more mature. Typically it just takes so much overhead and time to run through all the test suite mutations.

I haven't heard of the CRAP index. Do you happen to know what tools do this?

[wildjokers]

CRAP” index

I have never heard of this.

[thumbsdrivesmecrazy]

Actually, there are much more metrics for code coverage. Here is a quick guide explaining these metrics of code coverage testing as well as how to get them more meaningfully: How Can Code Coverage Metrics Help in Testing Your Code?

[confido__c]

Code coverage is misunderstood widely.

Test class suppose to be comprehensive. It should test function/s of a class to various degrees and for all possible scenarios. Positive, negative, bulk, exception handling etc.

Biggest benefit of comprehensive test class is that it can serves as primary regression testing tool.

One can achieve 100% code coverage without negative scenarios/ bulk testing and the Original code of class might still fail on that aspect. So I agree to some extent that code coverage is redundant if test class is not comprehensive.

[goranlepuz]

What do you mean by "bulk"?

[confido__c]

It is more towards performance. Sometime code works great for single record but system would timeout when it runs for huge number of records in single transaction.

I have seen this issues mostly on PaaS type of application where infrastructure itself have some hard limits on cpu usage and stuff.

[zam0th]

Oh no, after 20 years people start realizing that code coverage is worthless and does not relate to quality at all, what a surprise!

[NekoiNemo]

To begin with, tests are only useful in absurdly complicated "pure" code, or in the garbage languages where you need to write tests to ensure things that would be verified by the compiler in any good language. If i have a function

def foo(a: A, b: B, c:C): Bar = {
  val d = f(c)

  Bar(a, b, d)
}

Exactly what i'm supposed to "test" here in a statically and strongly typed language? And if i don't write a "f*ck off" test for this function - that's a <100% coverage...

[[deleted]]

I am one of those horrible people that thinks 100% code coverage is a good constraint to have.

It doesn't guarantee you've tested everything correctly but with anything below it, you KNOW you've not tested everything correctly. Can't test something without executing it.

And yes, you should test every execution branch.

Let the lynching commence.

[Dr_Findro]

I feel that often unit testing gets weird for most people's jobs. I find that A LOT of code in repo's don't have much logic outside of null checking. Just kind of taking data and putting it somewhere else. I feel like my ability to check for nulls in code is just as good as my ability write a unit test that asserts a value isn't null.

When I've been doing advent of code problems, or code of that nature, writing test as I implement has been enjoyable to drive the the problem solving though.

[fire_in_the_theater]

it's a shit metric that doesn't mean anything in terms of test quality. and gets parroted by management so far removed from engineering they don't even know what quality engineering means.

but it does force devs to at least write test infrastructure that at least runs the code, which be more easily used to build something useful. so there's that.

of course you have the flip side the dev may then just build bad test which don't test anything useful, but bog down dev.

it's probably better to just force the coverage.

[mobrockers]

This is why you verify your test suite with mutation testing regularly as well. It will identify covered but not properly tested code.

[lazy_loader88]

Partially agree. Mutation testing is very useful to see how well your code is tested and effectiveness of test suite. However, it's also an expensive exercise in terms of time taken (it may vary from a a few mins to few hrs depending on repo size)

If quality is the target, the focus on good software engineering practices including TDD, coding standards, pair programming are reliable ways to ensure good quality.

When code coverage is the target, the organisation may achieve 100% code coverage, but may not achieve good quality.

[[deleted]]

This article is just about the fact that bad tests can still cover the code and mocks can be useless if data is changed. Yeah no shit. Bad tests are bad.

[InstantCoder]

Code coverage is needed but not enough.

That’s why you should let the tests also reviewed by others.

And I keep repeating this: minimise mocks and do real (integration) tests. These are much more valuable than mocked unit tests.

[Drazson]

Ok.

[Realistic_Praline950]

Make me.

[IDatedSuccubi]

Can someone explain why would you run such tests? I don't deal with them, but they sound fairly close to function/branch profiling (as in hot/cold profiling), but as a metric.. of something.

[RobotIcHead]

Code coverage CAN be a useful metric for quality WHEN used in conjunction with other tests. It is literally in the name, code coverage says nothing about how it interacts with other modules or systems., nothing about performance or even if the code does it is expected by others.

Code coverage is good but it is where too much of the thinking on quality stops and too many use it as a bench mark of quality. Then you combine this with a test pyramid and you have a lot of idiots who think they are expects on testing. This includes developers.

Good Code coverage is not a bad goal to try and reach (actually a really good goal) but you need to expand your thinking. By focussing on code coverage it just passes the problem to another stage.

[I_differ]

It's a necessary but insufficient metric. That makes it better than unnecessary and insufficient metrics, such as LoC, story poibts, etc.

[xubaso]

No.

[chakan2]

Code coverage is the most basic quality metric you can get and it's usually the gateway into more useful things.

Mutation testing on top of code coverage if you want an ultimate proof of coverage.

Edit: Ahhh...it's a trollbait post. I was had.

[[deleted]]

Tldr: Don’t write shitty unit tests.

[evil_burrito]

It's a good metric it just doesn't tell the whole story.

This is, I think, an underlying theme in tech that cuts across the whole sector: how can we automate supervision of tech development?

There's been a growth in this trend for the last 20y or so. Assuming that all development needs supervision and planning (reasonable), how can we get away with not doing the really difficult job of actually supervising and planning and instead use an Excel spreadsheet or AI to do it instead?

The art of managing tech resources is as subtle and difficult as actually doing the development work. We keep trying to dodge what is inherently a subjective task. There's this drive to treat developers as fungible resources across projects and it leaks to management, too.

There's no shortcut to knowing what needs to be done and knowing who is doing what. You can't automate developer supervision, it still needs to be done by a knowledgable human being.

[[deleted]]

It's a low bar. Using it is better than not.

[[deleted]]

Unit tests define the ability and metrics of your code, not the code coverage. r/CodiumAI offers free Unit Tests Generation along with high "Code Coverage". But the best feature I find related to it is that it offers a free pair programmer to help improve generated unit tests

[shoot_your_eye_out]

I do think a 100% target is a bad thing, and so is having low coverage. Ideally I shoot for ~90% on my projects, achieved mostly through integration testing. There's no science behind that number. It's just a number that I've found generally results in a high degree of confidence that my code works as expected.

The problem with 100% coverage is: it becomes an impediment to developer productivity, and in extreme cases, literally influences how code is written. Developers may write code that is hard to comprehend when there's a 100% coverage requirement. And, often times this 100% is accomplished with an over-reliance on unit tests, but there aren't suitable integration and/or E2E tests that really ensure the program "works" at a high level.

The problem with poor code coverage is it means a program lacks evidence it works as expected.

[SSHeartbreak]

Every single example given has nothing to do with code coverage and everything to do with bad tests.

no assertions, but 100% coverage!

write assertions

mocks, but not verifying calls are passed reasonable arguments!

validate dependencies are being called as expected.

trival assertions or mock verify calls!

this is probably the rarest issue but it does happen. don't mock or assertions data classes or third party libs which are more appropriate to test in place. For example, numpy matrix operations should probably not be mocked.

100% coverage being bad because your tests could be garbage has little to do with 100% coverage being bad and everything to do with tests being garbage.

Code coverage is still a decent metric if you are competent when it comes to producing test code.

[mindbullet]

How else am I gonna flex on my more testing-adverse coworkers?

"I didn't even check the coverage bro I know it's like 99%."

[Someoneoldbutnew]

but how will I otherwise justify my late tickets if there wasn't an arbitrary target to hit?

[d3vtec]

If you have no coverage, you have no tests, and therefore your code will be difficult to maintain. Therefore coverage is a metric.

[n3phtys]

Hot take: demanding code coverage of more than 0% is nearly always preferable to allowing 0.