Add Software Bill of Materials (SBOM) for Python releases

[sethmlarson]

Feature or enhancement

Proposal:

Software Bill of Materials (SBOM) is a format for tracking software and its components. This information will also soon become relevant for Python users due to this Executive Order and other requirements elsewhere in the world. Instead of requiring each individual consumer and redistributor to create their own documents we can provide an authoritative document for each Python release.

This would not require a change to Python itself, instead I imagine the SBOM files would be provided alongside the release artifacts on python.org/downloads.

My goal with this project is to provide this information to consumers with minimal modification to core developer workflows. I've experimented with creating SBOMs for past and present Python versions and have found that most of the work comes when dependencies are updated and in those cases the SBOM metadata needs to also be updated (ie: versions, hashes). Beyond that the rest can be automated downstream with the Python release tooling.

I'm happy to make all the changes required to implement this proposal. I'm also happy to be the reviewer for all SBOM related PRs while I'm the Security Developer-in-Residence.

Proposed changes

• Create a file which tracks all bundled dependency paths and ignored files (ie Modules/_hacl/...)
• Add a new makefile target regen-sbom which regenerates the SBOM file containing hashes
• Run this target as a part of CI (via regen-all) to ensure that all updates to dependencies require an update to the SBOM metadata.

Then downstream in the release-tools repository:

• Grab the SBOM file for each tagged release to use as a base for each artifact
• Generate metadata for that Python release (files, relationships, etc).
• For each artifact, there may be specific dependencies pulled in that will be recorded (ie https://github.com/python/cpython-source-deps, https://github.com/python/cpython-bin-deps)
• Upload SBOM files to python.org/downloads similar to Sigstore signatures.

Example of updating dependencies

• Pull a new version of hacl-star, for example.
• make regen-all would cause changes to the checked in SBOM file. This would either fail in CI or require user to inspect the SBOM locally.
• Dev would read the instructions on how to update the version of the SBOM. Usually this would only require updating the version number and committing the generated changes to file checksums.
• Tool would check consistency of version information in other identifiers (PURL, CPE, download URL, etc)

Sub-issues

• Create an informational PEP on SBOMs in CPython
• Create a "What's New" entry for Python 3.13
• Add Software Bill-of-Materials (SBOM) documents and user guide to python.org/downloads pythondotorg#2339
• Create instructions for how to upgrade dependencies and keep SBOM up-to-date devguide#1241
• Add Software Bill-of-Materials for Windows source dependencies #112844
• Add Vulnerability Exchange (VEX) statements for CPython SBOMs to reference pythondotorg#2340

Has this already been discussed elsewhere?

I have already discussed this feature proposal on Discourse

Links to previous discussion of this feature:

I've created a Discourse topic to discuss the impact to core developers and maintenance.

Linked PRs

• gh-112302: Add Software Bill-of-Materials (SBOM) tracking for dependencies #112303
• gh-112302: Annotate SBOM file as generated in .gitattributes #112854
• gh-112302: Point core developers to SBOM devguide on errors #113490
• [3.12] gh-112302: Backport SBOM generation tooling #114730
• gh-112302: Change 'licenseConcluded' field to 'NOASSERTION' #115038
• [3.12] gh-112302: Change 'licenseConcluded' field to 'NOASSERTION' (GH-115038) #115088
• gh-112302: Move pip SBOM discovery to release-tools #115360
• [3.12] gh-112302: Move pip SBOM discovery to release-tools (GH-115360) #115486

[sethmlarson]

I've created a PR which ports the tooling used on my experimental repository into a regen-sbom makefile target.