Run all tests, but report partial coverage, for CI purposes

[kneczaj]

Clear and concise description of the problem

This is quite common need at CI pipeline that these two objectives need to be fulfilled:

1. run all tests
2. compute coverage of a pull request

The problem is that the second command reruns the same tests. When we talk about CI pipeline which does these two jobs, this is a lot of wasted time.

Suggested solution

My request is to add functionality to limit only collecting the coverage to what --changed (or similar) flag says, and still run the full test set. This could be e.g. --changed-coverage param.

Alternative

First

Currently this can be achieved by two separate commands, but takes up to twice as much time depending on changes:

1. run all tests: vitest run
2. compute coverage of a pull request: vitest run --coverage --changed=origin/main

Second

Compute coverage of all the project. Unfortunately this blurs the result as it includes a lot of files which are unrelated to pull request. This also is a problem at legacy projects which are in the process of increasing test coverage as unrelated files which may have no coverage are mixed with affected files with missing coverage.

Additional context

This is performance improvement for CI pipelines which use vitest

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

[AriPerkkio]

Could this be achieved already using vitest run --coverage --coverage.include=<here list of files changed in the pull request>. You could use something like https://github.com/marketplace/actions/changed-files or parse the git diff output manually.

[kneczaj]

Could this be achieved already using vitest run --coverage --coverage.include=<here list of files changed in the pull request>. You could use something like https://github.com/marketplace/actions/changed-files or parse the git diff output manually.

It cannot, as what I see, when --coverage --changed is used, it includes also all the files imported by changed files. This is what I look for to check coverage not only on all changed files, but on all affected files. The workaround here would include code parsing and imports analysis what is too much to consider this as a legit workaround.

[AriPerkkio]

I think this feature makes sense. So to run all tests but get coverage only based on changed files.

This could be test.coverage.changed, which gets its default value from test.changed.