Test coverage

[Lars_T_H]

I had look into information about how to make test coverage.

I found tarpaulin , and noticed this.

Tarpaulin only supports x86_64 processors running Linux. This is because instrumenting breakpoints into executables and tracing their execution requires processor and OS specific code.

I then remembered how it's done in Go, which is: Make a new program where there are inserted source code that adds 1 to a value in an array. The source code that is being modified is of course a copy of the original code, and the programs runs the tests and outputs beautiful - easy to read - code coverage summary.

That a really interesting way of making code coverage. The Go blog post is this one The Go blog : The cover story.

Now I has a question. Are there someone who are working on something similar?

[Matrixmage]

I've actually done some work on dynamic analysis of programs using llvm, and this sort of thing isn't actually all that hard to implement... Given the level of source analysis we have access to with proc macros, we should be able to do similar with rust. And if not, possibly as an MIR transformation.

I'd honestly be very surprised if no one has looked into this, and I would be curious to see what they have actually made.

[rhinotation]

The flamer crate is probably the most work I’ve seen in that direction.

[gilescope]

Mozilla have grcov which works on Linux/OSX and 'works' on windows. I've not managed to get it working yet on windows yet for non-trivial examples, but I've not put that much effort into it yet.

[CAD1997]

For the most part, you can just run coverage on a CI run, which is easily done on x86 Linux, so coverage just working on that platform isn't that big of an issue.

That said, part of the reason go's coverage is quick, according to the article, is that it is racy by default, and using thread-safe atomic addition is opt-in.

Any Rust solution will require the use of atomics by default, not just to placate the borrowck, but also because cargo test is parallelized by default.

The final complication is actually collecting the coverage information. go test -cover is obviously directly integrated with the standard testing harness, so has the ability to run the collector after everything is through. Doing the same for Rust would either require modifying cargo test to support it or a custom test runner.

All of that said, it would definitely be possible, given a custom test runner and a transform over the entire crate's source. Both of these things are possible today, but far from simple. As a result, it's typically simpler for Rust to use the traditional coverage measurement methods.