Test Coverage
We know that test coverage does not guarantee bug free code. But we do believe that lack of coverage is almost a guarantee for bugs!
Hence we have tried hard to achieve a high test coverage for the decimal4j library. We are testing all arithmetic operations, all scales, all rounding modes etc. Most of our tests operate on two test sets: random tests and special case tests.
The random tests simply generate random input values to test a certain function. We usually assert our actual results against the expected value derived from an equivalent operation of the BigDecimal (the comprehensiveness of our testing has lead to finding and reporting JDK bugs in the BigDecimal class!)
The special case tests are constructed from special input values such as zero, one, minus one, powers of 10, integer and long min/max values etc. We also randomly choose mutable and immutable Decimal values and alter between equivalent alternatives to perform the tested operation.
All of this does not come for free: a full coverage build takes 13 hours on our build server! This is why we have introduced different test variants for decimal4j builds. For more information on test variants see next section.
Test variants are controlled via system properties when running the build, e.g.
gradle clean build jacocoTestReport -DtestVariant=SMALL
| Feature / Variant | TINY | SMALL | STANDARD | LARGE | ALL |
|---|---|---|---|---|---|
| Build Duration | <2min | ~3min | ~5min | 20-30min | 12-13h |
| Tested Scales | 0,9,18 | 0,6,9,18 | 0,6,9,18 | 0,6,9,17,18 | 0-18 |
| Rounding Modes | DOWN, HALF_UP | DOWN, HALF_UP | DOWN, HALF_UP, HALF_EVEN | UP, DOWN, HALF_UP, HALF_EVEN, UNNEC. | (all) |
| Rounding Modes + Overflow Check | DOWN, UNNEC. | DOWN, HALF_UP, UNNEC. | DOWN, HALF_UP, UNNEC. | UP, DOWN, HALF_UP, HALF_EVEN, UNNEC. | (all) |
| Random Tests* | 100 | 1000 | 1000 | 10000 | 10000 |
| Special Values** | 43 | 67 | 99 | 171 | 231 |
* Random test count per unit test
** Note that the number of special cases has a big impact on the build performance because the number is squared for operations with two arguments!
Test scales can be controlled separately, to run STANDARD test with ALL scales:
gradle clean build jacocoTestReport -DtestScales=ALL
Test cases can also be chosen separately, which impacts the special cases and # of random tests:
gradle clean build jacocoTestReport -DtestCases=ALL
Rounding modes and overflow modes are determined by the test variant only.