Request to support a "--cov-fail-under" option

[pandichef]

In pytest, I can add an option called --cov-fail-under that will throw an error to Linux if my test coverage isn't above some threshold. I use this to enforce a high test coverage percentage for CI (via Github Actions).

Can such an option be added to solidity-coverage? If this is currently possible to achieve in some other way, please advise.

I'm using the Hardhat plugin FWIW.

Thanks!

[cgewecke]

Hi @pandichef

There is a way.

Coverage writes a coverage.json file to your project's root directory.

istanbul (one of solidity-coverage's deps) has a check-coverage cli command which does the same thing as --cov-fail-under and is used like this:

$ npx istanbul check-coverage ./coverage.json --statements 99 --branches 94 --functions 99 --lines 99

ERROR: Coverage for statements (60%) does not meet global threshold (99%)
ERROR: Coverage for lines (60%) does not meet global threshold (99%)
ERROR: Coverage for functions (66.67%) does not meet global threshold (99%)

Agree this would be nice to have as a built-in option here though...

[pandichef]

Amazing! Thank you!

Is there a way to limit check-coverage to a subset of the project's contracts?

[cgewecke]

@pandichef

I think this would need to be done by post-processing the coverage.json file, deleting the entries you'd like to ignore.

It's a JSON object with outer level keys like:

{
  "contracts/Greeter.sol" : {
    // data
  },
  "contracts/AnotherContract.sol": {
    // etc
  }
}

You can also specify files and folders to ignore using solidity-coverage's skipFiles config option. No coverage data is collected for files listed in that array.

[cgewecke]

Have added documentation about this in #834. I don't think this will be added as a built-in feature so closing for now.