"Coverage Drift" protection doesn't seem to work

[maxp-hover]

We are setting up coverband and trying to think ahead about any edge cases that we may encounter. One of those cases would be "coverage drift". I have read the issue thread #118 and taken a look at the merged PR #137

I have been running some tests and it doesn't seem the "coverage drift" protection is working. If I call a method in a file, change its content, perform some action within it again, and then report the coverage, it shows line usage on lines which are impossible to run (on whitespace, or end lines, etc). It doesn't seem like the changed MD5 hash is invalidating the file in the redis db, or performing any kind of clearing of the file's state.

We are using HashRedisStore by the way. Would this have any impact on whether or not "coverage drift" protection works? I see in the above linked issues some mention of a "time since last changed" attribute, did this make it to the light of day?

We perform daily deploys and so it isn't feasible for us to clear the coverage DB every deploy. Ideally we would clear the coverage data for only those files which were changed, and be able to see when was the last time they were cleared.

Best regards Max

[kbaum]

HashRedisStore was intended to take md5 hash of the file into account.

https://github.com/danmayer/coverband/blob/master/lib/coverband/adapters/hash_redis_store.rb#L180
https://github.com/danmayer/coverband/blob/master/test/coverband/adapters/hash_redis_store_test.rb#L119

Are you testing in development? If so are you restarting your instance when you change a file? Thought is that maybe caching of the md5 per file is getting in the way?

[maxp-hover]

@kbaum Yes I am testing in development, and not restarting when I change. I didn't consider that it was caching the hash. It does seem to be working when I restart the server. The Coverage first seen number changes to something more recent. Thanks! I will close the issue.

[kbaum]

Great to hear that worked! Another gotcha to add to the README :).

[danmayer]

Yeah if in dev, this would be an issue. Sorry for the confusion @maxp-hover I guess we could check the config config.cache_classes and if it is not set (it is false in default dev) never cache, but that would slow down development... so I am not sure it is worth it as this is kind of a weird dev testing edge case.

I do think we could add a 'dev testing gotcha' to the readme as @kbaum suggested