Collect coverage information with a separate coverage build

[mikea]

Current source level coverage is often confusing. Primarily because of instrumentation optimizations and instrumenting edges rather than basic blocks.

Collect coverage information from a separate build configuration built without sanitizers, with -O1 -fsanitize-coverage=bb and sanitizer-coverage-prune-block disabled.

[kcc]

-mllvm -sanitizer-coverage-prune-block will need to be turned into a -fsanitize-coverage-something flag
(we should not be using -mllvm flags outside of llvm tests)

Also need to try various strategies to eliminate unreachable code (including -ffunction-sections and -flto)

[mikea]

This change (https://reviews.llvm.org/rL290017) forces sancov to ignore dead files during load time. This should a) reduce sizes of individual .symcov files b) reduce total memory footprint.

@oliverchang please try again with fresh version of sancov

[oliverchang]

I hacked together something to build with clang's coverage instrumentation (https://clang.llvm.org/docs/SourceBasedCodeCoverage.html): oliverchang@343ddff.

With this we can get coverage like: https://oss-fuzz-build-logs.storage.googleapis.com/test/json.html

llvm-cov also has an export option to export the coverage data to a json format, similar to symcov. This seems promising, but I'm not sure how well this format scales compared to symcov. For json parse_afl_fuzzer, it's already more than 4x the size.

[kcc]

This is very nice, and might be a better way to go than to extend sanitizer-coverage visualization.

4x might be ok, as long as it's not quadratic or something :)

[kcc]

I've just implemented one more -fsanitize-coverage= flag: no-prune.
With this sub-flag we disable coverage pruning, i.e. actually instrument every basic block or edge.
Combined with bb instead of the default edge visualizing the lines of code should be less confusing.
Thus, if you use -fsanitize-coverage=bb,no-prune,trace-pc-guard the quality of coverage visualization should go up.

(It still needs to be a separate build, because if we combine no-prune with actual fuzzing we will slow down things)

[oliverchang]

Thanks, I'll set up another build next week to experiment with this.

[oliverchang]

This is done. We'll have results across more projects in 1-2 days, but here are some preliminary ones

freetype2: old vs new

json: old vs new