Generate gcov coverage data

[steveklabnik]

Issue by brson
Thursday Jul 14, 2011 at 17:33 GMT

For earlier discussion, see rust-lang/rust#690

This issue was labelled with: A-an-interesting-project, A-debuginfo, A-testsuite in the Rust repository

---

It would be nice to know what our test coverage looks like. I believe this depends on rust-lang/rust#689

[reem]

Relevant hyper issue (linking): hyperium/hyper#44

[lifthrasiir]

Kcov had an issue with position-independent executables that rustc generates by default. This should be fixed by a PR at SimonKagstrom/kcov#77. (EDIT: This is now merged, upcoming kcov v26 or later should be fine.)

[farcaller]

Just for the reference, SimonKagstrom/kcov#77 allows to get coverage output from a casual binary produced by cargo build/test (on linux host only, obviously).

[shahn]

Recently a compiler-rt commit removed the explicit profile targets, and I'm not having any luck with just adding the required functions to FUNCTIONS.builtins in make/platform/triple.mk: compiler-rt builds and seems usable, but rustc segfaults when using -C passes=insert-gcov-profiling

[shahn]

Turns out the segfault is fixed by rust-lang/rust#31176 - so this might again be a possibility to investigate!

[frewsxcv]

@shahn Have you tried running insert-gcov-profiling after rust-lang/rust#31176 got merged?

[shahn]

Yes, but I don't have it working (yet, maybe?). Currently fighting with llvm (https://llvm.org/bugs/show_bug.cgi?id=26294)

[shahn]

So, I have it working locally. It requires a few things: A custom-built (nightly) rustc and compiler-rt with support for profiling, not using more than one codegen-units, adding the insert-gcov-profiling pass to the rustc invocation as well as linking with the custom compiler-rt.
I also had to symlink the stdlib source into my project's src directory because I couldn't figure out how to ignore these. Another issue happened around the lazy_static crate I used, because lcov really wanted a file <lazy_static macros> to exist in my project's top level directory.

The generated coverage data looks mostly ok, but has some strange things:

• it reports all derives as unused
• in all struct declarations, it reports the declaration of members as unused
• it reports "pub mod xy" lines as unused

The picture gets a bit worse when looking at branch coverage, which seems basically unusable for Rust in its current state. use statements sometimes creates tons of branches, mostly in never-executed basic blocks, but sometimes even in executed basic blocks. No idea what's up there. Generally rust code seems to be very branch heavy.

[shahn]

Oh, one more thing. One cannot use gcc's gcov tool to interpret the data depending on gcc version, but rather llvm-cov (to be built from the make generated during rustc's build process) because the data format varies wildly between gcov versions. gcov from new GCC just segfaults when given such a gcda file.

What's the process to proceed here? I would love to add support for this to be easier and actually be eventually shipped by default with rustc (I'm thinking "cargo coverage" or something). This would need build system changes, I have absolutely zero idea about windows compatibility (it works on OSX and Linux), and also an idea how to distribute the necessary compiler-rt as well as llvm-cov (perhaps relabeled to rustcov or something to avoid name clashes with a concurrently installed llvm). Is it worthwhile to start the process of implementing the necessary changes?

[nrc]

@shahn Could you link to your branch please? It would be good to see what changes are necessary to rustc.

The ideal way forward for this is to have a crate which does it outside of the main Rust repo and to develop it there until it is a bit more mature. Deep changes to the compiler should land if needed, but a lot of stuff can be done by using the compiler as a library and wrapping calls to it. You can also make Cargo plugins to experiment with cargo coverage. If there is not much to do other than compiler changes then the next step would be to put a PR together and discuss on that. If it is a lot of changes, you could make a compiler RFC, but that is probably not necessary.

This sounds really cool! Thanks for investigating it.