Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test:Code coverage report #1159

Merged
merged 2 commits into from
Sep 4, 2020
Merged

test:Code coverage report #1159

merged 2 commits into from
Sep 4, 2020

Conversation

marcotc
Copy link
Member

@marcotc marcotc commented Aug 28, 2020

Seeing is believing: https://84200-66497320-gh.circle-artifacts.com/0/coverage/index.html

Screen Shot 2020-08-28 at 5 43 12 PM

This PR adds code coverage reporting for our CI tests.

Tests from different runs are merged together in a new coverage step.
A report is made available as an uploaded artifact in the coverage step, under coverage/index.html:
Screen Shot 2020-08-28 at 5 44 54 PM

It's also possible to see the individual coverage on a per Ruby version basis: these reports will be uploaded on the same build step, under (e.g. for Ruby 2.0) coverage/versions/test-2.0/index.html. This is specially useful when adding support for a new Ruby version.

A report can also be generated locally by executing bundle exec rake coverage:report after running any of our test suites or individual tests. Results will be written to the ./coverage/ directory.

@marcotc marcotc requested a review from a team August 28, 2020 21:49
@marcotc marcotc self-assigned this Aug 28, 2020
@marcotc marcotc added dev/ci Involves CircleCI, GitHub Actions, or GitLab dev/testing Involves testing processes (e.g. RSpec) labels Aug 28, 2020
@brettlangdon
Copy link
Member

Will this get reported on the PR? How can we be sure people actually look at it?

@marcotc
Copy link
Member Author

marcotc commented Aug 31, 2020

@brettlangdon We don't have any plans to enforce coverage right now.

The tools that allow us to report coverage back to the PR (with a PR comment most likely) require coverage to be collected first, which is the scope of this PR.

In the future, we can use SaaS to handle the Github updates, or push them back manually ourserlves from CircleCI back to Github by giving it proper credentials.

@marcotc
Copy link
Member Author

marcotc commented Aug 31, 2020

I'm going to add documentation to our DevelopmentGuide.md, as that's the most appropriate location today.

I'll look into adding a pull request template with this information too soon.

Copy link
Contributor

@ericmustin ericmustin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's worth thinking about how we may enforce this in the future, lgtm though

@marcotc marcotc merged commit ee2cc2d into master Sep 4, 2020
@marcotc marcotc deleted the test/coverage-report branch September 4, 2020 19:28
@marcotc marcotc added this to the 0.40.0 milestone Sep 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dev/ci Involves CircleCI, GitHub Actions, or GitLab dev/testing Involves testing processes (e.g. RSpec)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants