use md5 to avoid coverage drift on changes

[danmayer]

OK @dnasseri and @kbaum this takes the ideas from #128 but pulls them into Coverband 3 and pushes as much of the logic to the adapter base class, as we don't want to have to do much specific to any given adapter.

I believe this should resolve the drift issue: #118

• I need to do a bit more work and review of the benchmarks as I think this does slow things down.
• The report doesn't yet show the additional time information, about the timerange valid for a given file, that will take a bit more work and another PR, but I think we could likely get this in without that.
  • I am thinking to do that I will need to drop direct dependencies on Simplecov / Simplecov-html

[kbaum]

Very excited for this one!

[danmayer]

OK, I am hoping to hear back on my other PR about supporting both aws-sdk versions and on the issues, Jared was having with app files not getting tracked... but I think we might be able to cut an alpha version of 3.1 this week with your background work and the MD5 work. I will do a bit more cleanup and testing.

[danmayer]

ok tiny perf fix, but now the perf looks good with little to no difference... So I am merging this one in.

3.0.0
Coverage report generated for Unit Tests to /Users/danmayer/projects/coverband/coverage. 356 / 429 LOC (82.98%) covered.
runs benchmarks on reporting large sets of files to redis
Warming up --------------------------------------
       store_reports     2.000  i/100ms
Calculating -------------------------------------
       store_reports     27.962  (± 7.2%) i/s -    420.000  in  15.069795s
comparing Coverage loaded/not, this takes some time for output...
coverage loaded
Coverband configured with file store
Warming up --------------------------------------
           coverband     2.000  i/100ms
Calculating -------------------------------------
           coverband     22.284  (±13.5%) i/s -    262.000  in  12.024206s

Pausing here -- run Ruby again to measure the next benchmark...
just the work
Coverband configured with file store
Warming up --------------------------------------
        no coverband     2.000  i/100ms
Calculating -------------------------------------
        no coverband     21.492  (±27.9%) i/s -    230.000  in  12.021139s

Comparison:
           coverband:       22.3 i/s
        no coverband:       21.5 i/s - same-ish: difference falls within error

comparing Coverage loaded/not, this takes some time for output...
coverage loaded
Coverband configured with default Redis store
Warming up --------------------------------------
           coverband     1.000  i/100ms
Calculating -------------------------------------
           coverband     13.624  (± 7.3%) i/s -    163.000  in  12.068790s

Pausing here -- run Ruby again to measure the next benchmark...
just the work
Coverband configured with default Redis store
Warming up --------------------------------------
        no coverband     2.000  i/100ms
Calculating -------------------------------------
        no coverband     21.624  (±23.1%) i/s -    238.000  in  12.008669s

Comparison:
        no coverband:       21.6 i/s
           coverband:       13.6 i/s - 1.59x  slower


--- vs md5 version ---

runs benchmarks on reporting large sets of files to redis
Warming up --------------------------------------
       store_reports     2.000  i/100ms
Calculating -------------------------------------
       store_reports     25.173  (± 4.0%) i/s -    378.000  in  15.061486s
comparing Coverage loaded/not, this takes some time for output...
coverage loaded
Coverband configured with file store
Warming up --------------------------------------
           coverband     2.000  i/100ms
Calculating -------------------------------------
           coverband     20.799  (±19.2%) i/s -    238.000  in  12.071631s

Pausing here -- run Ruby again to measure the next benchmark...
just the work
Coverband configured with file store
Warming up --------------------------------------
        no coverband     2.000  i/100ms
Calculating -------------------------------------
        no coverband     21.275  (±32.9%) i/s -    226.000  in  12.051278s

Comparison:
        no coverband:       21.3 i/s
           coverband:       20.8 i/s - same-ish: difference falls within error

comparing Coverage loaded/not, this takes some time for output...
coverage loaded
Coverband configured with default Redis store
Warming up --------------------------------------
           coverband     1.000  i/100ms
Calculating -------------------------------------
           coverband     15.213  (±13.1%) i/s -    180.000  in  12.058447s

Pausing here -- run Ruby again to measure the next benchmark...
just the work
Coverband configured with default Redis store
Warming up --------------------------------------
        no coverband     2.000  i/100ms
Calculating -------------------------------------
        no coverband     21.258  (±32.9%) i/s -    222.000  in  12.012867s

Comparison:
        no coverband:       21.3 i/s
           coverband:       15.2 i/s - same-ish: difference falls within error

---
AB tests 3.0.0

Requests per second:    9.43 [#/sec] (mean)
Time per request:       1060.721 [ms] (mean)
Time per request:       106.072 [ms] (mean, across all concurrent requests)
Transfer rate:          52.86 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   118 1053  86.2   1053    1287
Waiting:      117 1052  85.9   1052    1258
Total:        118 1053  86.1   1053    1288

Percentage of the requests served within a certain time (ms)
  50%   1053
  66%   1079
  75%   1094
  80%   1104
  90%   1136
  95%   1158
  98%   1181
  99%   1207
 100%   1288 (longest request)

--- vs 3.0.1 with md5 -------

Requests per second:    9.01 [#/sec] (mean)
Time per request:       1109.506 [ms] (mean)
Time per request:       110.951 [ms] (mean, across all concurrent requests)
Transfer rate:          50.53 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       1
Processing:    93 1101  97.0   1104    1301
Waiting:       93 1100  96.8   1102    1301
Total:         93 1102  96.9   1104    1302

Percentage of the requests served within a certain time (ms)
  50%   1104
  66%   1136
  75%   1153
  80%   1162
  90%   1199
  95%   1221
  98%   1260
  99%   1266
 100%   1302 (longest request)