Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(gazelle): runfiles discovery #1429

Merged
merged 2 commits into from
Sep 26, 2023
Merged

fix(gazelle): runfiles discovery #1429

merged 2 commits into from
Sep 26, 2023

Conversation

aignas
Copy link
Collaborator

@aignas aignas commented Sep 22, 2023

Pass the environment generated by the runfiles helper so that the
Python launcher can find the runfiles correctly as implemented in
bazelbuild/bazel#14740.

Fixes #1426

@aignas
Copy link
Collaborator Author

aignas commented Sep 22, 2023

@fmeum, could you please take a look? This is an additional change needed on top of your work in #1415.

I have tested it with:

cd examples/bzlmod_build_file_generation
USE_BAZEL_VERSION=last_green bazel run //:gazelle

@aignas aignas changed the title fix(gazelle): fix runfiles discovery fix(gazelle): runfiles discovery Sep 22, 2023
@fmeum
Copy link
Contributor

fmeum commented Sep 22, 2023

Just FYI: With bazel-contrib/bazel-gazelle#1632, the Gazelle wrapper will be setting these variables and this change should only matter for tests at that point.

@rickeylev
Copy link
Contributor

I'm not familiar with Go, so its not clear to me why this works. All it seems to be doing is passing Env() to the subprocess? I don't see Env being modified. Do subprocesses in Go not inherit the current env? I sort of expected to see the PYTHON_RUNFILES envvar being set.

gazelle/python/parser.go Outdated Show resolved Hide resolved
@fmeum
Copy link
Contributor

fmeum commented Sep 24, 2023

I'm not familiar with Go, so its not clear to me why this works. All it seems to be doing is passing Env() to the subprocess? I don't see Env being modified. Do subprocesses in Go not inherit the current env? I sort of expected to see the PYTHON_RUNFILES envvar being set.

rfiles.Env() discovers runfiles (potentially by looking next to argv[0]) and then returns the environment variables that subprocesses need to discover runfiles. If no runfiles env variables are set for the outer process, the inner process wouldn't be able to discover runfiles without this line (which has a bug, see my comment).

@rickeylev
Copy link
Contributor

Thanks. LGTM after Fabian's comments are addressed.

@aignas
Copy link
Collaborator Author

aignas commented Sep 25, 2023

I am going to leave this like this due to my observation in #1429 (comment). @rickeylev, @f0rmiga, feel free to merge it during your working hours if you agree to postpone including os.Env() in the std_module.go.

@aignas aignas added this pull request to the merge queue Sep 26, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Sep 26, 2023
@aignas aignas added this pull request to the merge queue Sep 26, 2023
Merged via the queue into main with commit 48daf52 Sep 26, 2023
5 checks passed
renovate bot referenced this pull request in bazel-contrib/rules_bazel_integration_test Oct 6, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [rules_python](https://togithub.com/bazelbuild/rules_python) |
http_archive | minor | `0.25.0` -> `0.26.0` |

---

### Release Notes

<details>
<summary>bazelbuild/rules_python (rules_python)</summary>

###
[`v0.26.0`](https://togithub.com/bazelbuild/rules_python/releases/tag/0.26.0)

[Compare
Source](https://togithub.com/bazelbuild/rules_python/compare/0.25.0...0.26.0)

#### Using Bzlmod with Bazel 6

**NOTE: bzlmod support is still beta. APIs subject to change.**

Add to your `MODULE.bazel` file:

```starlark
bazel_dep(name = "rules_python", version = "0.26.0")

pip = use_extension("@&#8203;rules_python//python/extensions:pip.bzl", "pip")

pip.parse(
    name = "pip",
    requirements_lock = "//:requirements_lock.txt",
)

use_repo(pip, "pip")
```

#### Using WORKSPACE

Paste this snippet into your `WORKSPACE` file:

```starlark
load("@&#8203;bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_python",
    sha256 = "9d04041ac92a0985e344235f5d946f71ac543f1b1565f2cdbc9a2aaee8adf55b",
    strip_prefix = "rules_python-0.26.0",
    url = "https://github.com/bazelbuild/rules_python/releases/download/0.26.0/rules_python-0.26.0.tar.gz",
)

load("@&#8203;rules_python//python:repositories.bzl", "py_repositories")

py_repositories()
```

##### Gazelle plugin

Paste this snippet into your `WORKSPACE` file:

```starlark
load("@&#8203;bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
    name = "rules_python_gazelle_plugin",
    sha256 = "9d04041ac92a0985e344235f5d946f71ac543f1b1565f2cdbc9a2aaee8adf55b",
    strip_prefix = "rules_python-0.26.0/gazelle",
    url = "https://github.com/bazelbuild/rules_python/releases/download/0.26.0/rules_python-0.26.0.tar.gz",
)

### To compile the rules_python gazelle extension from source,
### we must fetch some third-party go dependencies that it uses.

load("@&#8203;rules_python_gazelle_plugin//:deps.bzl", _py_gazelle_deps = "gazelle_deps")

_py_gazelle_deps()
```

#### What's Changed

- doc: Note Python version changes in CHANGELOG by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1391](https://togithub.com/bazelbuild/rules_python/pull/1391)
- fix: bcr releaser email by
[@&#8203;f0rmiga](https://togithub.com/f0rmiga) in
[https://github.com/bazelbuild/rules_python/pull/1392](https://togithub.com/bazelbuild/rules_python/pull/1392)
- Adding kwargs to gazelle_python_manifest by
[@&#8203;linzhp](https://togithub.com/linzhp) in
[https://github.com/bazelbuild/rules_python/pull/1289](https://togithub.com/bazelbuild/rules_python/pull/1289)
- docs: Use correct link to build badge image and build status page. by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1390](https://togithub.com/bazelbuild/rules_python/pull/1390)
- feat(py_console_script_binary)!: entry points with custom dependencies
by [@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/1363](https://togithub.com/bazelbuild/rules_python/pull/1363)
- fix(whl_library): avoid unnecessary repository rule restarts by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/1400](https://togithub.com/bazelbuild/rules_python/pull/1400)
- refactor: add missing `//python/config_settings/private:distribution`
target by [@&#8203;philsc](https://togithub.com/philsc) in
[https://github.com/bazelbuild/rules_python/pull/1402](https://togithub.com/bazelbuild/rules_python/pull/1402)
- Import pycross_wheel_library by
[@&#8203;philsc](https://togithub.com/philsc) in
[https://github.com/bazelbuild/rules_python/pull/1403](https://togithub.com/bazelbuild/rules_python/pull/1403)
- refactor: upgrade certifi by
[@&#8203;cflewis](https://togithub.com/cflewis) in
[https://github.com/bazelbuild/rules_python/pull/1397](https://togithub.com/bazelbuild/rules_python/pull/1397)
- fix: don't set distribs in version transitioning rule by
[@&#8203;comius](https://togithub.com/comius) in
[https://github.com/bazelbuild/rules_python/pull/1412](https://togithub.com/bazelbuild/rules_python/pull/1412)
- fix(gazelle): upgrade rules_go: 0.39.1 -> 0.41.0 to work with upcoming
Bazel versions by [@&#8203;sgowroji](https://togithub.com/sgowroji) in
[https://github.com/bazelbuild/rules_python/pull/1410](https://togithub.com/bazelbuild/rules_python/pull/1410)
- fix: gazelle: Fix non-hermetic runfiles lookup by
[@&#8203;fmeum](https://togithub.com/fmeum) in
[https://github.com/bazelbuild/rules_python/pull/1415](https://togithub.com/bazelbuild/rules_python/pull/1415)
- feat: create toolchain type for py_proto_library by
[@&#8203;comius](https://togithub.com/comius) in
[https://github.com/bazelbuild/rules_python/pull/1416](https://togithub.com/bazelbuild/rules_python/pull/1416)
- internal: copy Starlark rule implementation from Bazel by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1418](https://togithub.com/bazelbuild/rules_python/pull/1418)
- feat: add new Python toolchain versions by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/1414](https://togithub.com/bazelbuild/rules_python/pull/1414)
- internal(pystar): make starlark impl (mostly) loadable by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1422](https://togithub.com/bazelbuild/rules_python/pull/1422)
- feat: generate py_library per file by
[@&#8203;raylu](https://togithub.com/raylu) in
[https://github.com/bazelbuild/rules_python/pull/1398](https://togithub.com/bazelbuild/rules_python/pull/1398)
- chore: bump default python versions by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/1425](https://togithub.com/bazelbuild/rules_python/pull/1425)
- feat: Support netrc-based authentication for python_repository rule by
[@&#8203;LINKIWI](https://togithub.com/LINKIWI) in
[https://github.com/bazelbuild/rules_python/pull/1417](https://togithub.com/bazelbuild/rules_python/pull/1417)
- refactor(pystar): load (but don't use) Starlark implementation. by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1428](https://togithub.com/bazelbuild/rules_python/pull/1428)
- fix(gazelle): runfiles discovery by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/1429](https://togithub.com/bazelbuild/rules_python/pull/1429)
- feat, refactor(pystar): bzl_library for packaging.bzl; fix pystar doc
building and py_wheel by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1432](https://togithub.com/bazelbuild/rules_python/pull/1432)
- refactor(toolchain): use a helper method to convert an X.Y version to
X.Y.Z by [@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/1423](https://togithub.com/bazelbuild/rules_python/pull/1423)
- pycross: Rename `pycross_wheel_library` and make it work by
[@&#8203;philsc](https://togithub.com/philsc) in
[https://github.com/bazelbuild/rules_python/pull/1413](https://togithub.com/bazelbuild/rules_python/pull/1413)
- fix: Skip printing unneccesary warning. by
[@&#8203;matts1](https://togithub.com/matts1) in
[https://github.com/bazelbuild/rules_python/pull/1407](https://togithub.com/bazelbuild/rules_python/pull/1407)
- refactor(bzlmod)!: simplify pip.parse repository layout by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/1395](https://togithub.com/bazelbuild/rules_python/pull/1395)
- feat(bzlmod): mark pip extension as os/arch dependent by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/1433](https://togithub.com/bazelbuild/rules_python/pull/1433)
- chore: bump internal_deps by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/1322](https://togithub.com/bazelbuild/rules_python/pull/1322)
- tests(pystar): CI configs that uses Starlark implementation of rules
by [@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1435](https://togithub.com/bazelbuild/rules_python/pull/1435)
- internal(pystar): Copy @&#8203;bazel_tools//tools/python files to
rules_python by [@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1437](https://togithub.com/bazelbuild/rules_python/pull/1437)
- internal(pystar): Make py_runtime_pair and autodetecting toolchain
mostly loadable. by [@&#8203;rickeylev](https://togithub.com/rickeylev)
in
[https://github.com/bazelbuild/rules_python/pull/1439](https://togithub.com/bazelbuild/rules_python/pull/1439)
- tests: Move base rule tests under tests instead of
//tools/build_defs/python by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1440](https://togithub.com/bazelbuild/rules_python/pull/1440)
- tests(pystar): py_runtime_pair and py_runtime analysis tests by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1441](https://togithub.com/bazelbuild/rules_python/pull/1441)
- fix(pystar): Use py_internal for runfiles_enabled,
declare_shareable_artifact, share_native_deps by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1443](https://togithub.com/bazelbuild/rules_python/pull/1443)
- build(deps): bump urllib3 from 1.26.13 to 1.26.17 in
/examples/pip_repository_annotations by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/bazelbuild/rules_python/pull/1447](https://togithub.com/bazelbuild/rules_python/pull/1447)
- build(deps): bump urllib3 from 1.25.11 to 1.26.17 in
/examples/pip_install by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/bazelbuild/rules_python/pull/1444](https://togithub.com/bazelbuild/rules_python/pull/1444)
- fix: add missing `@bazel_tools` files to bzl_library dependencies. by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1457](https://togithub.com/bazelbuild/rules_python/pull/1457)
- tests(pystar): add analysis tests to cover basic windows building by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1452](https://togithub.com/bazelbuild/rules_python/pull/1452)
- docs: move dependency management into respective bzl packages by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1459](https://togithub.com/bazelbuild/rules_python/pull/1459)
- feat(py_wheel): Normalize name and version by
[@&#8203;vonschultz](https://togithub.com/vonschultz) in
[https://github.com/bazelbuild/rules_python/pull/1331](https://togithub.com/bazelbuild/rules_python/pull/1331)
- chore: add new Python toolchains from indygreg by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/1461](https://togithub.com/bazelbuild/rules_python/pull/1461)

#### New Contributors

- [@&#8203;cflewis](https://togithub.com/cflewis) made their first
contribution in
[https://github.com/bazelbuild/rules_python/pull/1397](https://togithub.com/bazelbuild/rules_python/pull/1397)
- [@&#8203;sgowroji](https://togithub.com/sgowroji) made their first
contribution in
[https://github.com/bazelbuild/rules_python/pull/1410](https://togithub.com/bazelbuild/rules_python/pull/1410)
- [@&#8203;raylu](https://togithub.com/raylu) made their first
contribution in
[https://github.com/bazelbuild/rules_python/pull/1398](https://togithub.com/bazelbuild/rules_python/pull/1398)
- [@&#8203;LINKIWI](https://togithub.com/LINKIWI) made their first
contribution in
[https://github.com/bazelbuild/rules_python/pull/1417](https://togithub.com/bazelbuild/rules_python/pull/1417)

**Full Changelog**:
bazelbuild/rules_python@0.25.0...0.26.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/bazel-contrib/rules_bazel_integration_test).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4wLjMiLCJ1cGRhdGVkSW5WZXIiOiIzNy4wLjMiLCJ0YXJnZXRCcmFuY2giOiJtYWluIn0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
@aignas aignas deleted the fix/1426/gazelle-python branch January 14, 2024 10:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

AssertionError: Cannot find .runfiles directory
4 participants