feature-request: first-class test-coverage support

[bcoe]

I find code-coverage is an extremely valuable tool for ensuring that a codebase is well tested. It's also a great way to figure out performance bottlenecks.

newer versions of v8 have built in code coverage, through the inspector API:

https://v8project.blogspot.com/2017/12/javascript-code-coverage.html

It would be really nice if we could figure out a way to expose this functionality out of the gate in deno, with source-map support; something I've been trying to do retroactively in Node.js.

[bcoe]

CC: @schuay, @TimothyGu, @addaleax

[kt3k]

other references:

• Code Coverage directly from V8
• c8
• Rethinking JavaScript Test Coverage

From nodejs/node

• coverage: expose native V8 coverage coverage: expose native V8 coverage nodejs/node#22527
• coverage: make coverage work for Node.js coverage: make coverage work for Node.js nodejs/node#23941

[kitsonk]

This is really interesting, and bothered me how we would try to accomplish this. Great reminder of the capabilities of V8.

We would need to take the coverage information and for transpiled modules, remap via the source maps. We have the logic in Rust to deal with the source maps and so it should be workable. I assume we would want to output a coverage.json ala Istanbul.js and then we could benefit from the nyc tooling to transform to other formats of coverage analysis.

[bcoe]

@kitsonk see https://github.com/bcoe/c8, which can turn raw v8 output into Istanbul reports.

[kitsonk]

Thinking about this after a while. We need a few things:

• Access to the inspector: APIs required to implement inspector rusty_v8#205
• Use/port: v8-to-istanbul
• Integration into the testing framework
• Ensure that reporters and the like for Istanbul.js work under Deno.

[kt3k]

FYI Rome seems doing something similar on their own. https://github.com/facebookexperimental/rome/blob/master/packages/%40romejs/v8/CoverageCollector.ts

[bartlomieju]

I tentatively started working on this issue - managed to get coverage data when running deno test.

First #4780 must be fixed; secondly data from V8 must be parsed into human readable form.

I like approach that rome took which is completely self-contained solution with text report to console. Parsing coverage data to Rust structs means JSON output is pretty much free (thanks serde); so I suggest starting with our own solution and thinking about integration with other tools after we have working MVP.