Coverage reporting refactor and cleanup by Cadair · Pull Request #397 · OpenAstronomy/github-actions-workflows

Cadair · 2026-06-03T10:00:48Z

fixes #395 and fixes #383

This PR makes the following changes:

Breaking change you need to set setenv = COVERAGE_FILE={toxinidir}/.coverage in tox.ini if your tests are not run in the {toxinidir} directory, so that reporting finds the .coverage file.
coverage: codecov no longer requires the use of pytest-cov, the xml coverage report is generated using coverage.py in a separate step to the tox run.
We now support OIDC auth to codecov meaning you don't need CODECOV_TOKEN if using this.
coverage: github is now tested against both pytest-cov and coverage.py, and as long as the .coverage files are in the root directory will work correctly.
As discussed in BUG: Coverage that works for v2.4.0 breaks after v2.5.0 #383 (comment) codecov will now fail the job on error, including no files updated. This is particularly critical with the breaking change.

codecov-commenter · 2026-06-03T11:29:02Z

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

astrofrog

This seems ok to me! Did you test it out on a real package? If so feel free to merge.

No combining may be needed

Cadair · 2026-06-03T12:23:09Z

I'm having trouble with it on sunpy: sunpy/sunpy#8665

Cadair · 2026-06-03T12:45:36Z

Fixed sunpy: https://github.com/sunpy/sunpy/actions/runs/26883832859

Cadair · 2026-06-03T15:46:17Z

Do people think it's worth throwing a warning about the COVERAGE_FILE change if we get to the codecov upload step and there isn't a .coverage file to work with?

pllim · 2026-06-03T15:58:51Z

Do people think it's worth throwing a warning

I would say, yes. Not everyone follows the PRs here or even read the change log.

Cadair · 2026-06-03T16:51:49Z

Ok, done, and failing test added to demo. I think this is ready, I'll put out a 3.0 after this.

pllim

I have a comment and a question.

pllim · 2026-06-03T17:05:41Z

So you just gonna leave the py314-tmpdir_warning job red?

Cadair · 2026-06-03T17:12:16Z

So you just gonna leave the py314-tmpdir_warning job red?

yep lol

Co-authored-by: P. L. Lim <2090236+pllim@users.noreply.github.com>

pllim · 2026-06-03T18:10:59Z

as per OpenAstronomy/github-actions-workflows#397

* Bump the actions group in /.github/workflows with 6 updates Bumps the actions group in /.github/workflows with 6 updates: | Package | From | To | | --- | --- | --- | | [actions/checkout](https://github.com/actions/checkout) | `6.0.2` | `7.0.0` | | [actions/setup-python](https://github.com/actions/setup-python) | `6.2.0` | `6.3.0` | | [actions/cache](https://github.com/actions/cache) | `5.0.5` | `6.1.0` | | [OpenAstronomy/github-actions-workflows/.github/workflows/tox.yml](https://github.com/openastronomy/github-actions-workflows) | `2.6.3` | `3.0.1` | | [OpenAstronomy/github-actions-workflows/.github/workflows/publish.yml](https://github.com/openastronomy/github-actions-workflows) | `2.6.3` | `3.0.1` | | [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) | `8.1.0` | `8.2.0` | Updates `actions/checkout` from 6.0.2 to 7.0.0 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@de0fac2...9c091bb) Updates `actions/setup-python` from 6.2.0 to 6.3.0 - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@a309ff8...ece7cb0) Updates `actions/cache` from 5.0.5 to 6.1.0 - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@27d5ce7...55cc834) Updates `OpenAstronomy/github-actions-workflows/.github/workflows/tox.yml` from 2.6.3 to 3.0.1 - [Release notes](https://github.com/openastronomy/github-actions-workflows/releases) - [Commits](OpenAstronomy/github-actions-workflows@2835f0c...e5af21e) Updates `OpenAstronomy/github-actions-workflows/.github/workflows/publish.yml` from 2.6.3 to 3.0.1 - [Release notes](https://github.com/openastronomy/github-actions-workflows/releases) - [Commits](OpenAstronomy/github-actions-workflows@2835f0c...e5af21e) Updates `astral-sh/setup-uv` from 8.1.0 to 8.2.0 - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](astral-sh/setup-uv@0880764...fac544c) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: 7.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/setup-python dependency-version: 6.3.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: actions - dependency-name: actions/cache dependency-version: 6.1.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: OpenAstronomy/github-actions-workflows/.github/workflows/tox.yml dependency-version: 3.0.1 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: OpenAstronomy/github-actions-workflows/.github/workflows/publish.yml dependency-version: 3.0.1 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: astral-sh/setup-uv dependency-version: 8.2.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: actions ... Signed-off-by: dependabot[bot] <support@github.com> * TST: Define COVERAGE_FILE in tox.ini as per OpenAstronomy/github-actions-workflows#397 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: P. L. Lim <2090236+pllim@users.noreply.github.com>

Cadair force-pushed the fixup_gh_cov branch 3 times, most recently from de82f8e to 6912510 Compare June 3, 2026 10:16

Cadair added 2 commits June 3, 2026 11:17

Add a test for pytest-cov with github coverage

6059407

Don't require suffix on .coverage file

ad8caed

Cadair force-pushed the fixup_gh_cov branch from 6912510 to ad8caed Compare June 3, 2026 10:17

Add a test for running in a tmpdir

793a91e

Cadair force-pushed the fixup_gh_cov branch 4 times, most recently from 62bc579 to 231bbd9 Compare June 3, 2026 10:46

Cadair added 2 commits June 3, 2026 12:00

Improve documentation around coverage

781f3a5

Set COVERAGE_FILE in tox.ini

4265ace

Cadair force-pushed the fixup_gh_cov branch 2 times, most recently from 8b40a0f to ff18080 Compare June 3, 2026 11:27

Cadair force-pushed the fixup_gh_cov branch from ff18080 to b8913ec Compare June 3, 2026 11:36

Cadair added 4 commits June 3, 2026 12:37

Add some codecov tests

3db5cbc

Fail CI if codecov upload fails

55d5f2f

Explicitly generate an xml report for codecov

881bb20

Support oidc auth for codecov

6e95ce5

Cadair force-pushed the fixup_gh_cov branch from b8913ec to 6e95ce5 Compare June 3, 2026 11:37

Cadair marked this pull request as ready for review June 3, 2026 11:44

Cadair changed the title ~~Attempts at smoothing the migration path to coverage:github~~ Coverage reporting refactor and cleanup Jun 3, 2026

Cadair requested review from ConorMacBride, astrofrog and zacharyburnett June 3, 2026 11:45

Cadair added 2 commits June 3, 2026 12:47

BREAKING CHANGE: Remove --cov-report pytest flag

33e0dad

Try to reproduce sunpy fail

63806a1

astrofrog approved these changes Jun 3, 2026

View reviewed changes

Cadair added 2 commits June 3, 2026 13:14

Add docs on source mapping for coverage: github

5dfa618

Ignore errors in coverage combine

fc76dfa

No combining may be needed

Cadair force-pushed the fixup_gh_cov branch from 4f6a343 to fc76dfa Compare June 3, 2026 12:20

Split combine and report

9d321c7

pllim reviewed Jun 3, 2026

View reviewed changes

Comment thread tox.ini Outdated

zacharyburnett approved these changes Jun 3, 2026

View reviewed changes

Comment thread .github/workflows/tox.yml

Comment thread .github/workflows/tox.yml Outdated

Some more tweaks

e0c57e9

Cadair force-pushed the fixup_gh_cov branch from 24827d8 to e0c57e9 Compare June 3, 2026 16:48

Cadair mentioned this pull request Jun 3, 2026

Do not explicitly specify uv version in calls to setup-uv #396

Merged

Cadair requested a review from pllim June 3, 2026 16:51

pllim reviewed Jun 3, 2026

View reviewed changes

Comment thread docs/source/tox.rst Outdated

Cadair and others added 2 commits June 3, 2026 17:12

Update docs/source/tox.rst

a1cc672

Co-authored-by: P. L. Lim <2090236+pllim@users.noreply.github.com>

Merge branch 'main' into fixup_gh_cov

70f406f

Cadair merged commit e113052 into OpenAstronomy:main Jun 3, 2026
90 of 91 checks passed

Cadair deleted the fixup_gh_cov branch June 3, 2026 19:36

This was referenced Jun 3, 2026

replace fill: parameter with logic to expand py3* glob syntax in envs #392

Merged

Better Reporting of Coverage Reporting Errors #399

Merged

DEP: upgrade cibuildwheel to 4.0.0 #393

Merged

pllim added a commit to astropy/astropy that referenced this pull request Jul 1, 2026

TST: Define COVERAGE_FILE in tox.ini

2f99416

as per OpenAstronomy/github-actions-workflows#397

neutrinoceros mentioned this pull request Jul 29, 2026

CI: coverage combining is broken #416

Open

Uh oh!

Conversation

Cadair commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Jun 3, 2026

Welcome to Codecov 🎉

Uh oh!

astrofrog left a comment

Choose a reason for hiding this comment

Uh oh!

Cadair commented Jun 3, 2026

Uh oh!

Cadair commented Jun 3, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Cadair commented Jun 3, 2026

Uh oh!

pllim commented Jun 3, 2026

Uh oh!

Cadair commented Jun 3, 2026

Uh oh!

pllim left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pllim commented Jun 3, 2026

Uh oh!

Cadair commented Jun 3, 2026

Uh oh!

pllim commented Jun 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Cadair commented Jun 3, 2026 •

edited

Loading