Adding a 'checkCoverage' option to compare coverage with a defined threshold

[sebarmeli]

At the moment it's not possible to run the 'check-coverage' command available in Istanbul. It would be cool to have a way of specifying thresholds for statement coverage, function coverage and branch coverage.

I was thinking something like

checkCoverage: {
   statement : 90,
   branch: 80,
   function: 100
}

to be added to the Karma conf file.

Thoughts?

[xzyfer]

+1

[roman01la]

👍

[rpocklin]

👍 and bump instanbul to 0.2.7 so I can generate clover reports :)

[andersekdahl]

👍

[aymericbeaumet]

@rpocklin I bumped Istanbul to 0.2.10 (#78), you will have it on the next release of this plugin.

[bfowle]

+1

[andrewconnell]

+1 bump

[aymericbeaumet]

I'm not that much involved in this plugin development anymore, but feel free to PR guys. I'll review it.

[stramel]

+1

[stramel]

This will accomplish what you guys are looking for. Would be nice to add this to the documentation. First number is the threshold for Red to Yellow, second number is the threshold for Yellow to Green

// @File: karma.config.js

coverageReporter: {
  type : 'html',
  dir: 'reports',
  subdir: 'coverage',
  watermarks: {
    statements: [50,75],
    lines: [50,75],
    functions: [50,75],
    branches:[50,75]
  }
},

[peter-mouland]

@stramel if the thresholds are red, is there a way for the task to fail?

[stramel]

@peter-mouland Good question! I would have to look into it.

[peter-mouland]

thanks for the interest, i've done it quite crudely with gulp tasks. unfortunately i dont have the time to go through the coverage plug and see how to do it properly. hopefully one day....

gulp.task('test:single-run', function (done) {
    karma.start({
        configFile: findup(paths.test.config),//path to  karma.conf.js
        singleRun: true
    }, done);
});
gulp.task('test', ['test:single-run'], function(cb){
    var results = require(findup(paths.test.summary)); //path to the json-summary.json set in karma.conf.js
    var config = require(findup(paths.test.config));//path to  karma.conf.js
    var coverage = config({set: function(conf){return conf;}}).coverageReporter;
    var thresholds = coverage.reporters[0].watermarks; //path to watermarks within for type: html
    var err = false;
    for (var file in results){
        for (var threshold in thresholds){
            if (results[file][threshold].pct < thresholds[threshold][0]){
                handleError(file + ' : ' + threshold + ' Coverage is too low (<' + thresholds[threshold][0] + '%)');
                err = true;
            }
        }
    }
    if (err) process.exit(1); //exit once all errors have been gathered
    cb();
});