Problem
When a project is built with a PEP 517 frontend with build isolation (pip, uv, build), the frontend creates a randomly-named temporary environment and maturin points PyO3 at that environment's interpreter:
maturin pep517 build-wheel -i /home/runner/.cache/uv/builds-v0/.tmpyHvXPy/bin/python
pyo3-build-config's build script registers cargo:rerun-if-env-changed=PYO3_PYTHON. That path is inside the temporary virtualenv, so its different on every build, and cargo recompiles pyo3-build-config, pyo3-ffi, pyo3, pyo3-macros-backend, and pyo3-macros on every pip/uv install ., even when the interpreter, version, and ABI are identical and the cargo target dir is fully warm.
For any project that caches the target dir in CI (e.g. Swatinem/rust-cache), this defeats the cache. Measured on pyca/cryptography's CI: every run recompiled the five pyo3 crates on every platform; on Windows it added ~45s per job.
Proposed Solution
My proposal is two parts:
- Change
maturin to export a PYO3_BASE_PYTHON env var (in addition to PYO3_PYTHON) which points to the stable python binary path (i.e. the non-venv path, basically sys._base_executable)
- Change
pyo3-build-config to prefer reading PYO3_BASE_PYTHON if it's available
This should give you: pyo3-build-config has a stable path for metadata purposes, and PYO3_PYTHON is still available if you need it.
What do folks think? Should be easy to do PRs for this, and it should rollout fine:
| maturin |
pyo3 |
behavior |
| old |
old |
status quo |
| new |
old |
extra env var ignored; status quo |
| old |
new |
variable absent; status quo |
| new |
new |
pyo3 chain stays cached across PEP 517 builds |
cc @messense
Problem
When a project is built with a PEP 517 frontend with build isolation (pip, uv,
build), the frontend creates a randomly-named temporary environment and maturin points PyO3 at that environment's interpreter:pyo3-build-config's build script registers
cargo:rerun-if-env-changed=PYO3_PYTHON. That path is inside the temporary virtualenv, so its different on every build, and cargo recompilespyo3-build-config,pyo3-ffi,pyo3,pyo3-macros-backend, andpyo3-macroson everypip/uv install ., even when the interpreter, version, and ABI are identical and the cargo target dir is fully warm.For any project that caches the target dir in CI (e.g.
Swatinem/rust-cache), this defeats the cache. Measured on pyca/cryptography's CI: every run recompiled the five pyo3 crates on every platform; on Windows it added ~45s per job.Proposed Solution
My proposal is two parts:
maturinto export aPYO3_BASE_PYTHONenv var (in addition toPYO3_PYTHON) which points to the stable python binary path (i.e. the non-venv path, basicallysys._base_executable)pyo3-build-configto prefer readingPYO3_BASE_PYTHONif it's availableThis should give you:
pyo3-build-confighas a stable path for metadata purposes, andPYO3_PYTHONis still available if you need it.What do folks think? Should be easy to do PRs for this, and it should rollout fine:
cc @messense