Measure test coverage

[JamieMagee]

Code improvement description

Test coverage is a simple measure of how much of the codebase has been tested. I think that setting specific coverage levels is not very helpful beyond a certain point, but it can help identify parts of the codebase that are poorly tested and may require more attention.

The most popular library for measuring code coverage in Ruby is simplecov.

[yeikel]

Can you assign this to me?

[JamieMagee]

@yeikel done. Thank you!

[smoookeeey]

Can you assign this to me?

[JamieMagee]

@smoookeeey I assigned you too, but @yeikel asked to be assigned first. Please make sure you don't duplicate any work @yeikel might have already done.

[smoookeeey]

Yes I will make sure not too do that

[yeikel]

I haven't started yet, and I won't have capacity until sometime next week. @smoookeeey, if you pick it up before that, feel free to create the PR and link it here

[smoookeeey]

I will look into it brother

[abdulapopoola]

Thanks a lot @yeikel and @smoookeeey !

[JamieMagee]

I just discovered that test coverage was previously added in #2576 to prevent a regression. Originally line coverage was set to 80% and branch coverage to 70%, with refuse_coverage_drop enabled to prevent any drops from occuring.

It was later gated behind the COVERAGE environment variable in #3339. However, because that environment variable wasn't set in GitHub Actions, code coverage was allowed to deteriorate. Subsequently, test coverage was removed in #7473.

[yeikel]

I think that rather than enforcing a certain %, we can start reporting it for awareness

[JamieMagee]

I agree. Reporting-only is a good start. Then something like refuse_coverage_drop to prevent backslides and act like a ratchet

[yeikel]

@smoookeeey Are you still working on this? Feel free to create a draft PR whenever you're ready

[JamieMagee]

@yeikel I think you can go ahead with this if you want.

[JamieMagee]

@yeikel It didn't look like you were working on this, so I created #9595