Skip to content

Automate trusted package publishing - #1028

Merged
aallan merged 2 commits into
mainfrom
codex/release-trusted-publishing
Jul 15, 2026
Merged

Automate trusted package publishing#1028
aallan merged 2 commits into
mainfrom
codex/release-trusted-publishing

Conversation

@aallan

@aallan aallan commented Jul 15, 2026

Copy link
Copy Markdown
Owner

Summary

  • add a tested release-policy helper that detects version bumps, validates immutable releases, creates archive manifests, and verifies registry filenames and SHA-256 hashes
  • add a least-privilege GitHub Actions workflow for TestPyPI staging and approval-gated PyPI Trusted Publishing, followed by registry verification and the tag/GitHub Release
  • document one-time publisher/environment setup, ordinary releases, guarded recovery, and the immutable-release policy

Release behavior

This PR does not bump Vera's version and does not publish anything when merged. The first push of this workflow to main sees no version change and exits without building or publishing a release.

After merge, the maintainer still needs to create the testpypi and approval-protected pypi GitHub environments and configure the matching pending Trusted Publishers. The current 0.1.4 package can then be staged manually on TestPyPI. The first production publication remains 0.1.5, triggered by its eventual version-bump merge.

This completes the release-mechanics portion of the publication plan. Issue #737 remains open until veralang is live on PyPI and both documented installation routes have been verified.

Validation

  • full pre-commit suite
  • 7,351 passed, 70 skipped, 26 deselected
  • mypy vera/
  • ruff and formatting checks for the new helper/tests
  • actionlint for .github/workflows/release.yml
  • real wheel/sdist build, Twine check, distribution inspection, and installed-wheel CLI smoke outside the checkout
  • release-helper no-op push and explicit TestPyPI planning smoke tests
  • PyPI and TestPyPI absence checks for veralang==0.1.4

Closes #481.

Summary by CodeRabbit

  • New Features
    • Added an approval-gated automated release workflow for PyPI on new main version merges, including wheel/sdist building, smoke testing, and SHA-256 + filename verification.
    • Enforced immutable publishing rules (fails if versions/tags already exist) and creates Git tags/GitHub Releases only after successful verification.
    • Kept a separate, confirmation-based TestPyPI staging path.
  • Documentation
    • Expanded release automation, Trusted Publishing setup, recovery guidance, and immutability policy documentation.
  • Tests
    • Added comprehensive test coverage for release planning, changelog extraction, archive/hash generation, and registry verification.
  • Chores
    • Improved .venv entries in .gitignore.

Co-Authored-By: Claude <noreply@anthropic.invalid>
@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: ad42b938-f476-436c-8574-259fc2789789

📥 Commits

Reviewing files that changed from the base of the PR and between e77b97d and 9f65e47.

📒 Files selected for processing (6)
  • .github/workflows/release.yml
  • README.md
  • ROADMAP.md
  • TESTING.md
  • scripts/release.py
  • tests/test_release.py
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • aallan/vera-bench (manual)

📝 Walkthrough

Walkthrough

Changes

Release automation

Layer / File(s) Summary
Release planning and CLI
scripts/release.py, tests/test_release.py
Adds strict version, changelog, recovery, tag, and release-plan validation with CLI commands and comprehensive tests.
Archive and registry verification
scripts/release.py, tests/test_release.py
Builds deterministic SHA-256 manifests and verifies exact archive filenames and hashes against package registries.
Workflow preparation and build
.github/workflows/release.yml
Adds gated release triggers, preparation outputs, distribution builds, wheel smoke tests, and tested artifact uploads.
Publication and GitHub release
.github/workflows/release.yml
Publishes to TestPyPI or PyPI, verifies registry contents, and creates an immutable tag and GitHub Release.
Release policy and documentation
.gitignore, CHANGELOG.md, CLAUDE.md, CONTRIBUTING.md, README.md, RELEASING.md, ROADMAP.md, TESTING.md
Documents the release workflow and immutability policy, and updates project and testing metrics.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant MainPush
  participant GitHubActions
  participant ReleaseHelper
  participant Registry
  participant GitHubRelease
  MainPush->>GitHubActions: trigger release workflow
  GitHubActions->>ReleaseHelper: prepare and validate version
  ReleaseHelper-->>GitHubActions: target, version, and publish decision
  GitHubActions->>GitHubActions: build and smoke-test distributions
  GitHubActions->>Registry: publish tested archives
  GitHubActions->>ReleaseHelper: verify filenames and SHA-256 hashes
  ReleaseHelper-->>GitHubActions: registry verification
  GitHubActions->>GitHubRelease: create immutable tag and release
Loading

Suggested labels: ci, tests, docs

🚥 Pre-merge checks | ✅ 6 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning The PR also adds TestPyPI/production-recovery flows, a release helper, tests, and doc updates beyond #481’s auto-tag/release scope. Split the broader trusted-publishing and recovery machinery into a separate PR, or link the additional release-automation issue explicitly.
Docstring Coverage ⚠️ Warning Docstring coverage is 23.44% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (6 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title matches the main theme of automated trusted publishing and release automation.
Linked Issues check ✅ Passed The workflow now detects version bumps, validates sync, tags vX.Y.Z, and creates the GitHub Release with minimal permissions.
Changelog Covers Public-Surface Changes ✅ Passed No files in the listed public-surface areas changed; the only changelog entry covers release automation/docs work.
Spec And Implementation Move Together ✅ Passed No files under vera/ or spec/ changed in this PR, so there is no spec/implementation drift to check.
Diagnostics Carry An Error Code ✅ Passed The patch only touches release automation/docs/tests; no compiler diagnostics were added or changed, and scripts/release.py has no Diagnostic/error_code usage.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/release-trusted-publishing

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jul 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.51%. Comparing base (22f3236) to head (9f65e47).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1028   +/-   ##
=======================================
  Coverage   93.51%   93.51%           
=======================================
  Files          96       96           
  Lines       32024    32024           
  Branches      456      456           
=======================================
  Hits        29948    29948           
  Misses       2063     2063           
  Partials       13       13           
Flag Coverage Δ
javascript 78.41% <ø> (ø)
python 95.29% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
tests/test_release.py (1)

1-390: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

No coverage for main()'s CLI dispatch or _write_github_outputs.

Every public function in scripts/release.py is unit-tested directly except the main() subcommand wiring (prepare/notes/manifest/assert-absent/verify-registry) and _write_github_outputs. This is the actual seam the GitHub Actions workflow invokes, so an argparse wiring mistake or output-format regression here wouldn't be caught by the current suite.

def test_main_prepare_writes_github_outputs(tmp_path, monkeypatch):
    root = _root(tmp_path)
    monkeypatch.setattr(release, "version_at_ref", lambda _ref, _root: "0.1.4")
    monkeypatch.chdir(root)
    out = tmp_path / "gh_output"
    assert release.main(
        ["prepare", "--mode", "push", "--before-ref", "before", "--github-output", str(out)]
    ) == 0
    lines = out.read_text(encoding="utf-8").splitlines()
    assert "publish=true" in lines
    assert "target=pypi" in lines
    assert "artifact=veralang-0.1.5" in lines
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_release.py` around lines 1 - 390, Add CLI-level tests covering
main() dispatch for prepare, notes, manifest, assert-absent, and
verify-registry, plus _write_github_outputs formatting. Include a prepare
invocation that patches required release state, passes --github-output, asserts
main() returns 0, and verifies publish, target, and artifact entries in the
generated file; exercise the remaining subcommands sufficiently to catch
argparse wiring errors.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/release.yml:
- Around line 87-92: Update the four release workflow steps using
steps.testpypi.outputs.version or needs.prepare.outputs.version in run commands,
including the TestPyPI/PyPI absence checks and both verify-registry invocations.
Pass each interpolated version through the step’s env mapping, then reference
the environment variable in the shell command instead of embedding the GitHub
expression directly.
- Around line 202-224: Update the verify-testpypi and verify-pypi jobs to retry
registry verification while the published filenames or hashes are not yet
visible, allowing for index propagation after publishing. Keep verification
failures for persistent missing or mismatched artifacts distinct from transient
read-after-publish delays, and apply the same behavior to both registry checks
using the existing verify-registry command.

In `@scripts/release.py`:
- Around line 346-351: Update _write_github_outputs to build the artifact value
using the existing PROJECT constant instead of hardcoding “veralang”, while
preserving the current version interpolation and output format.

In `@tests/test_release.py`:
- Around line 288-312: Extend test_registry_version_files coverage for malformed
registry responses by asserting ReleaseError for a non-dict releases value, a
non-dict file entry, and entries missing filename or SHA-256 data. Use
pytest.raises with the corresponding error-message matches, while preserving the
existing happy-path and absent-project/version tests.

---

Outside diff comments:
In `@tests/test_release.py`:
- Around line 1-390: Add CLI-level tests covering main() dispatch for prepare,
notes, manifest, assert-absent, and verify-registry, plus _write_github_outputs
formatting. Include a prepare invocation that patches required release state,
passes --github-output, asserts main() returns 0, and verifies publish, target,
and artifact entries in the generated file; exercise the remaining subcommands
sufficiently to catch argparse wiring errors.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b664389b-ca02-4327-97c8-2c2fef727c01

📥 Commits

Reviewing files that changed from the base of the PR and between 22f3236 and e77b97d.

📒 Files selected for processing (11)
  • .github/workflows/release.yml
  • .gitignore
  • CHANGELOG.md
  • CLAUDE.md
  • CONTRIBUTING.md
  • README.md
  • RELEASING.md
  • ROADMAP.md
  • TESTING.md
  • scripts/release.py
  • tests/test_release.py
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • aallan/vera-bench (manual)

Comment thread .github/workflows/release.yml Outdated
Comment thread .github/workflows/release.yml
Comment thread scripts/release.py Outdated
Comment thread tests/test_release.py
Co-Authored-By: Claude <noreply@anthropic.invalid>
@aallan

aallan commented Jul 15, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai Addressed the outside-diff CLI coverage finding in 9f65e47.

tests/test_release.py now exercises:

  • _write_github_outputs formatting directly;
  • main() dispatch for prepare, including --github-output and the publish, target, version, and artifact entries;
  • notes output creation;
  • manifest generation from a real synthetic wheel/sdist pair;
  • assert-absent argument dispatch; and
  • verify-registry argument dispatch.

The review round also adds malformed registry-payload coverage and pins retry behavior for stale registry hashes. Validation: 47 focused release tests pass; ruff, formatting, actionlint, documentation counts/site assets, and all affected pre-commit gates pass, including the full 7,458-test suite.

@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
git log --oneline -1 9f65e472 2>/dev/null || echo "commit not found in shallow clone"
git show --stat 9f65e472 2>/dev/null | head -50
echo "---"
rg -n "_write_github_outputs|def test_main|assert-absent|verify-registry|--github-output" tests/test_release.py | head -80

Length of output: 3926


@aallan Confirmed — I checked commit 9f65e47 and tests/test_release.py now includes:

  • test_write_github_outputs (Line 443) exercising _write_github_outputs formatting directly
  • test_main_prepare_writes_github_outputs (Line 455), including the --github-output flag and dispatch through main()
  • test_main_notes (Line 486)
  • test_main_manifest (Line 496)
  • test_main_assert_absent (Line 508)
  • test_main_verify_registry (Line 521)

This closes the outside-diff CLI coverage gap I flagged — all the CLI subcommand entry points are now exercised via main() rather than only through the underlying helper functions. Thanks for the thorough follow-through, including the malformed-registry-payload and stale-hash retry pinning.

@aallan
aallan merged commit ce6e00e into main Jul 15, 2026
28 checks passed
@aallan
aallan deleted the codex/release-trusted-publishing branch July 15, 2026 19:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Auto-tag and auto-release on version bump in pyproject.toml

1 participant