Skip to content

Publish to PyPI via Trusted Publishing with PEP 740 attestations; retire PYPI_API_TOKEN #1958

Description

@oberstet

Problem

autobahn is the only WAMP Python package on PyPI without provenance. Its sibling packages
(txaio, zlmdb, cfxdb, xbr, crossbar) all upload through PyPI Trusted Publishing (OIDC) with
attestations: true, and PyPI serves a signed PEP 740 publish attestation for each of their files:

GET https://pypi.org/integrity/zlmdb/26.7.1/zlmdb-26.7.1.tar.gz/provenance
  -> publisher: GitHub crossbario/zlmdb, workflow release.yml, environment pypi
GET https://pypi.org/integrity/autobahn/26.7.1/autobahn-26.7.1-pp311-pypy311_pp73-win_amd64.whl/provenance
  -> 404 Not Found

For comparison, cryptography 50.0.1 carries an attestation on every file (in-toto statement,
Sigstore certificate, Rekor log entry, source commit and workflow run).

The Trusted Publisher is already registered on PyPI for autobahn
(crossbario/autobahn-python, release.yml, environment pypi), and the release-stable job
already has id-token: write and environment: pypi. But the upload step does not use either:

# .github/workflows/release.yml, job release-stable, "Publish to PyPI using bleeding-edge twine"
env:
  TWINE_USERNAME: __token__
  TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
run: |
  python3 -m pip install --break-system-packages git+https://github.com/pypa/packaging.git
  python3 -m pip install --break-system-packages git+https://github.com/pypa/twine.git
  twine upload ...

Two consequences:

  1. No provenance. Files are uploaded with a long-lived token, so PyPI cannot attest who
    published them. Attestations cannot be added to files after upload: every release until this is
    fixed stays unattested.
  2. Credential exposure. The job installs the current tip of two third-party repositories
    (pypa/twine, pypa/packaging) and runs them with the PyPI token in the environment (and with
    id-token: write in an environment: pypi job, so it could also mint a Trusted Publishing
    token). A malicious or compromised commit upstream would be enough to steal the credential or
    tamper with the upload. No autobahn project-scoped API token is listed on PyPI, so the
    PYPI_API_TOKEN secret is probably a user-scoped (account-wide) token that can upload to
    every project of that account.

The git-master installs were a workaround for PEP 639 (Core Metadata 2.4) support; released
versions of twine and packaging support it now (to be confirmed with the pinned versions).

Proposed change

  • Replace the twine upload step with pypa/gh-action-pypi-publish (pinned to a commit SHA), with
    attestations: true, exactly as in zlmdb / crossbar release.yml job release-production.
  • Remove TWINE_PASSWORD / secrets.PYPI_API_TOKEN from the workflow.
  • Replace every pip install git+https://github.com/pypa/{twine,packaging}.git in the workflows
    (release.yml release-development, release-nightly, release-stable; also wheels workflows)
    with pinned released versions. None of them may remain in the release-stable job.

Note: nothing has to be run to "produce" https://docs.pypi.org/attestations/publish/v1. That URL
is the in-toto predicate type of the attestation that gh-action-pypi-publish creates and uploads
itself when it runs under Trusted Publishing with attestations: true.

Provenance verification, the same in every WAMP Python project (shared script and recipe pattern
from wamp-proto/wamp-cicd, issue "verify-pypi"):

  • pypi-attestations added to the development tools installed by just install-tools.
  • A just verify-pypi [version] recipe that verifies the provenance of every file of a release
    on PyPI against https://github.com/crossbario/autobahn-python.
  • A "Verifying a release" section in docs/installation.rst showing the user-side command.

Maintainer actions (outside the repository)

  • After the first attested release: revoke the token behind PYPI_API_TOKEN on PyPI
    (Account settings, API tokens; check whether it is account-wide) and delete the secret from
    the repository (and the organisation, if it is also defined there).

Acceptance

  • The next autobahn release (26.9.1) serves a provenance document for every file:
    https://pypi.org/integrity/autobahn/<version>/<file>/provenance returns publisher
    crossbario/autobahn-python, workflow release.yml, environment pypi.
  • pypi-attestations verify pypi --repository https://github.com/crossbario/autobahn-python pypi:<file>
    succeeds for a wheel and the sdist.
  • The PyPI file page shows "Uploaded using Trusted Publishing? Yes".
  • grep -rn 'PYPI_API_TOKEN\|git+https://github.com/pypa' .github/workflows/ finds nothing.
  • just verify-pypi 26.9.1 passes; docs/installation.rst documents the check.
  • The publish job follows the fleet hardening (actions pinned by SHA, id-token: write only on
    that job).

Blocks: the 26.9.1 release (it must be the first attested autobahn release).

Related: wamp-proto/wamp-cicd issues "Harden the PyPI publish jobs fleet-wide" and "verify-pypi";
SLSA build provenance: #1842; .cicd/SLSA.md item 2.

Note: This issue was drafted with AI assistance (Claude Code).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions