fix(whl_library): avoid unnecessary repository rule restarts #1400
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Put the PYTHONPATH entries used in wheel building as a default value to a private attribute of the `whl_library` repository rule. This remove all but one restart and reduces the execution of ``` bazel clean --expunge --async && bazel build //entry_points:yamllint ``` from 50s to 45s.
This avoids extra calls to `rctx.path` which may cause a restart when fetching a repository. This remove the remaining restart and shaves another 2s.
This is not used anymore in our `whl_library`, so remove it. NOTE: if we think that there are uses outside our repo, then we should reconsider, but given that it is very trivial for the user to write the same helper, I would be inclined to reduce the API scope.
f0rmiga
approved these changes
Sep 4, 2023
renovate bot
referenced
this pull request
in bazel-contrib/rules_bazel_integration_test
Oct 6, 2023
[](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [rules_python](https://togithub.com/bazelbuild/rules_python) | http_archive | minor | `0.25.0` -> `0.26.0` | --- ### Release Notes <details> <summary>bazelbuild/rules_python (rules_python)</summary> ### [`v0.26.0`](https://togithub.com/bazelbuild/rules_python/releases/tag/0.26.0) [Compare Source](https://togithub.com/bazelbuild/rules_python/compare/0.25.0...0.26.0) #### Using Bzlmod with Bazel 6 **NOTE: bzlmod support is still beta. APIs subject to change.** Add to your `MODULE.bazel` file: ```starlark bazel_dep(name = "rules_python", version = "0.26.0") pip = use_extension("@​rules_python//python/extensions:pip.bzl", "pip") pip.parse( name = "pip", requirements_lock = "//:requirements_lock.txt", ) use_repo(pip, "pip") ``` #### Using WORKSPACE Paste this snippet into your `WORKSPACE` file: ```starlark load("@​bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") http_archive( name = "rules_python", sha256 = "9d04041ac92a0985e344235f5d946f71ac543f1b1565f2cdbc9a2aaee8adf55b", strip_prefix = "rules_python-0.26.0", url = "https://github.com/bazelbuild/rules_python/releases/download/0.26.0/rules_python-0.26.0.tar.gz", ) load("@​rules_python//python:repositories.bzl", "py_repositories") py_repositories() ``` ##### Gazelle plugin Paste this snippet into your `WORKSPACE` file: ```starlark load("@​bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") http_archive( name = "rules_python_gazelle_plugin", sha256 = "9d04041ac92a0985e344235f5d946f71ac543f1b1565f2cdbc9a2aaee8adf55b", strip_prefix = "rules_python-0.26.0/gazelle", url = "https://github.com/bazelbuild/rules_python/releases/download/0.26.0/rules_python-0.26.0.tar.gz", ) ### To compile the rules_python gazelle extension from source, ### we must fetch some third-party go dependencies that it uses. load("@​rules_python_gazelle_plugin//:deps.bzl", _py_gazelle_deps = "gazelle_deps") _py_gazelle_deps() ``` #### What's Changed - doc: Note Python version changes in CHANGELOG by [@​rickeylev](https://togithub.com/rickeylev) in [https://github.com/bazelbuild/rules_python/pull/1391](https://togithub.com/bazelbuild/rules_python/pull/1391) - fix: bcr releaser email by [@​f0rmiga](https://togithub.com/f0rmiga) in [https://github.com/bazelbuild/rules_python/pull/1392](https://togithub.com/bazelbuild/rules_python/pull/1392) - Adding kwargs to gazelle_python_manifest by [@​linzhp](https://togithub.com/linzhp) in [https://github.com/bazelbuild/rules_python/pull/1289](https://togithub.com/bazelbuild/rules_python/pull/1289) - docs: Use correct link to build badge image and build status page. by [@​rickeylev](https://togithub.com/rickeylev) in [https://github.com/bazelbuild/rules_python/pull/1390](https://togithub.com/bazelbuild/rules_python/pull/1390) - feat(py_console_script_binary)!: entry points with custom dependencies by [@​aignas](https://togithub.com/aignas) in [https://github.com/bazelbuild/rules_python/pull/1363](https://togithub.com/bazelbuild/rules_python/pull/1363) - fix(whl_library): avoid unnecessary repository rule restarts by [@​aignas](https://togithub.com/aignas) in [https://github.com/bazelbuild/rules_python/pull/1400](https://togithub.com/bazelbuild/rules_python/pull/1400) - refactor: add missing `//python/config_settings/private:distribution` target by [@​philsc](https://togithub.com/philsc) in [https://github.com/bazelbuild/rules_python/pull/1402](https://togithub.com/bazelbuild/rules_python/pull/1402) - Import pycross_wheel_library by [@​philsc](https://togithub.com/philsc) in [https://github.com/bazelbuild/rules_python/pull/1403](https://togithub.com/bazelbuild/rules_python/pull/1403) - refactor: upgrade certifi by [@​cflewis](https://togithub.com/cflewis) in [https://github.com/bazelbuild/rules_python/pull/1397](https://togithub.com/bazelbuild/rules_python/pull/1397) - fix: don't set distribs in version transitioning rule by [@​comius](https://togithub.com/comius) in [https://github.com/bazelbuild/rules_python/pull/1412](https://togithub.com/bazelbuild/rules_python/pull/1412) - fix(gazelle): upgrade rules_go: 0.39.1 -> 0.41.0 to work with upcoming Bazel versions by [@​sgowroji](https://togithub.com/sgowroji) in [https://github.com/bazelbuild/rules_python/pull/1410](https://togithub.com/bazelbuild/rules_python/pull/1410) - fix: gazelle: Fix non-hermetic runfiles lookup by [@​fmeum](https://togithub.com/fmeum) in [https://github.com/bazelbuild/rules_python/pull/1415](https://togithub.com/bazelbuild/rules_python/pull/1415) - feat: create toolchain type for py_proto_library by [@​comius](https://togithub.com/comius) in [https://github.com/bazelbuild/rules_python/pull/1416](https://togithub.com/bazelbuild/rules_python/pull/1416) - internal: copy Starlark rule implementation from Bazel by [@​rickeylev](https://togithub.com/rickeylev) in [https://github.com/bazelbuild/rules_python/pull/1418](https://togithub.com/bazelbuild/rules_python/pull/1418) - feat: add new Python toolchain versions by [@​aignas](https://togithub.com/aignas) in [https://github.com/bazelbuild/rules_python/pull/1414](https://togithub.com/bazelbuild/rules_python/pull/1414) - internal(pystar): make starlark impl (mostly) loadable by [@​rickeylev](https://togithub.com/rickeylev) in [https://github.com/bazelbuild/rules_python/pull/1422](https://togithub.com/bazelbuild/rules_python/pull/1422) - feat: generate py_library per file by [@​raylu](https://togithub.com/raylu) in [https://github.com/bazelbuild/rules_python/pull/1398](https://togithub.com/bazelbuild/rules_python/pull/1398) - chore: bump default python versions by [@​aignas](https://togithub.com/aignas) in [https://github.com/bazelbuild/rules_python/pull/1425](https://togithub.com/bazelbuild/rules_python/pull/1425) - feat: Support netrc-based authentication for python_repository rule by [@​LINKIWI](https://togithub.com/LINKIWI) in [https://github.com/bazelbuild/rules_python/pull/1417](https://togithub.com/bazelbuild/rules_python/pull/1417) - refactor(pystar): load (but don't use) Starlark implementation. by [@​rickeylev](https://togithub.com/rickeylev) in [https://github.com/bazelbuild/rules_python/pull/1428](https://togithub.com/bazelbuild/rules_python/pull/1428) - fix(gazelle): runfiles discovery by [@​aignas](https://togithub.com/aignas) in [https://github.com/bazelbuild/rules_python/pull/1429](https://togithub.com/bazelbuild/rules_python/pull/1429) - feat, refactor(pystar): bzl_library for packaging.bzl; fix pystar doc building and py_wheel by [@​rickeylev](https://togithub.com/rickeylev) in [https://github.com/bazelbuild/rules_python/pull/1432](https://togithub.com/bazelbuild/rules_python/pull/1432) - refactor(toolchain): use a helper method to convert an X.Y version to X.Y.Z by [@​aignas](https://togithub.com/aignas) in [https://github.com/bazelbuild/rules_python/pull/1423](https://togithub.com/bazelbuild/rules_python/pull/1423) - pycross: Rename `pycross_wheel_library` and make it work by [@​philsc](https://togithub.com/philsc) in [https://github.com/bazelbuild/rules_python/pull/1413](https://togithub.com/bazelbuild/rules_python/pull/1413) - fix: Skip printing unneccesary warning. by [@​matts1](https://togithub.com/matts1) in [https://github.com/bazelbuild/rules_python/pull/1407](https://togithub.com/bazelbuild/rules_python/pull/1407) - refactor(bzlmod)!: simplify pip.parse repository layout by [@​aignas](https://togithub.com/aignas) in [https://github.com/bazelbuild/rules_python/pull/1395](https://togithub.com/bazelbuild/rules_python/pull/1395) - feat(bzlmod): mark pip extension as os/arch dependent by [@​aignas](https://togithub.com/aignas) in [https://github.com/bazelbuild/rules_python/pull/1433](https://togithub.com/bazelbuild/rules_python/pull/1433) - chore: bump internal_deps by [@​aignas](https://togithub.com/aignas) in [https://github.com/bazelbuild/rules_python/pull/1322](https://togithub.com/bazelbuild/rules_python/pull/1322) - tests(pystar): CI configs that uses Starlark implementation of rules by [@​rickeylev](https://togithub.com/rickeylev) in [https://github.com/bazelbuild/rules_python/pull/1435](https://togithub.com/bazelbuild/rules_python/pull/1435) - internal(pystar): Copy @​bazel_tools//tools/python files to rules_python by [@​rickeylev](https://togithub.com/rickeylev) in [https://github.com/bazelbuild/rules_python/pull/1437](https://togithub.com/bazelbuild/rules_python/pull/1437) - internal(pystar): Make py_runtime_pair and autodetecting toolchain mostly loadable. by [@​rickeylev](https://togithub.com/rickeylev) in [https://github.com/bazelbuild/rules_python/pull/1439](https://togithub.com/bazelbuild/rules_python/pull/1439) - tests: Move base rule tests under tests instead of //tools/build_defs/python by [@​rickeylev](https://togithub.com/rickeylev) in [https://github.com/bazelbuild/rules_python/pull/1440](https://togithub.com/bazelbuild/rules_python/pull/1440) - tests(pystar): py_runtime_pair and py_runtime analysis tests by [@​rickeylev](https://togithub.com/rickeylev) in [https://github.com/bazelbuild/rules_python/pull/1441](https://togithub.com/bazelbuild/rules_python/pull/1441) - fix(pystar): Use py_internal for runfiles_enabled, declare_shareable_artifact, share_native_deps by [@​rickeylev](https://togithub.com/rickeylev) in [https://github.com/bazelbuild/rules_python/pull/1443](https://togithub.com/bazelbuild/rules_python/pull/1443) - build(deps): bump urllib3 from 1.26.13 to 1.26.17 in /examples/pip_repository_annotations by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/bazelbuild/rules_python/pull/1447](https://togithub.com/bazelbuild/rules_python/pull/1447) - build(deps): bump urllib3 from 1.25.11 to 1.26.17 in /examples/pip_install by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/bazelbuild/rules_python/pull/1444](https://togithub.com/bazelbuild/rules_python/pull/1444) - fix: add missing `@bazel_tools` files to bzl_library dependencies. by [@​rickeylev](https://togithub.com/rickeylev) in [https://github.com/bazelbuild/rules_python/pull/1457](https://togithub.com/bazelbuild/rules_python/pull/1457) - tests(pystar): add analysis tests to cover basic windows building by [@​rickeylev](https://togithub.com/rickeylev) in [https://github.com/bazelbuild/rules_python/pull/1452](https://togithub.com/bazelbuild/rules_python/pull/1452) - docs: move dependency management into respective bzl packages by [@​rickeylev](https://togithub.com/rickeylev) in [https://github.com/bazelbuild/rules_python/pull/1459](https://togithub.com/bazelbuild/rules_python/pull/1459) - feat(py_wheel): Normalize name and version by [@​vonschultz](https://togithub.com/vonschultz) in [https://github.com/bazelbuild/rules_python/pull/1331](https://togithub.com/bazelbuild/rules_python/pull/1331) - chore: add new Python toolchains from indygreg by [@​aignas](https://togithub.com/aignas) in [https://github.com/bazelbuild/rules_python/pull/1461](https://togithub.com/bazelbuild/rules_python/pull/1461) #### New Contributors - [@​cflewis](https://togithub.com/cflewis) made their first contribution in [https://github.com/bazelbuild/rules_python/pull/1397](https://togithub.com/bazelbuild/rules_python/pull/1397) - [@​sgowroji](https://togithub.com/sgowroji) made their first contribution in [https://github.com/bazelbuild/rules_python/pull/1410](https://togithub.com/bazelbuild/rules_python/pull/1410) - [@​raylu](https://togithub.com/raylu) made their first contribution in [https://github.com/bazelbuild/rules_python/pull/1398](https://togithub.com/bazelbuild/rules_python/pull/1398) - [@​LINKIWI](https://togithub.com/LINKIWI) made their first contribution in [https://github.com/bazelbuild/rules_python/pull/1417](https://togithub.com/bazelbuild/rules_python/pull/1417) **Full Changelog**: bazelbuild/rules_python@0.25.0...0.26.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/bazel-contrib/rules_bazel_integration_test). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4wLjMiLCJ1cGRhdGVkSW5WZXIiOiIzNy4wLjMiLCJ0YXJnZXRCcmFuY2giOiJtYWluIn0=--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
](https://renovatebot.com){kind=link}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Put the
PYTHONPATHentries used in wheel building as a default value to aprivate attribute of the
whl_libraryrepository rule and use resolved path ofthe interpreter target in creating execution environment to avoid repository
rule restarts when fetching external dependencies.
The extra private attribute on the
whl_libraryremoves all but one restartand the extra refactor removes the last restart observed when running, which
also reduces the total execution time from around 50s to 43s on my machine:
Fixes #1399