Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid resolving symbolic links when querying Python interpreters #11083

Merged
merged 5 commits into from
Jan 30, 2025

Conversation

zanieb
Copy link
Member

@zanieb zanieb commented Jan 29, 2025

Closes #11048

This brings the PythonEnvironment::from_root behavior in-line with the rest of uv Python discovery behavior (and in-line with pip). It's not clear why we were canonicalizing the path in the first place here.

@zanieb zanieb added the bug Something isn't working label Jan 29, 2025
@zanieb zanieb force-pushed the zb/fix-canonical branch 2 times, most recently from 3f60a60 to f83c9f9 Compare January 29, 2025 20:42
@zanieb
Copy link
Member Author

zanieb commented Jan 29, 2025

!!!

running 1 test
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Snapshot: run_linked_environment_path-2
Source: crates/uv/tests/it/run.rs:3395
────────────────────────────────────────────────────────────────────────────────
Expression: snapshot
────────────────────────────────────────────────────────────────────────────────
-old snapshot
+new results
────────────┬───────────────────────────────────────────────────────────────────
    0     0 │ success: true
    1     1 │ exit_code: 0
    2     2 │ ----- stdout -----
    3       │-[VENV]/
    4       │-[VENV]/[BIN]/python
          3 │+[TEMP_DIR]/target
          4 │+[TEMP_DIR]/target/[BIN]/python
    5     5 │ 
    6     6 │ ----- stderr -----
    7     7 │ Resolved 8 packages in [TIME]
    8     8 │ Audited 6 packages in [TIME]
────────────┴───────────────────────────────────────────────────────────────────
Stopped on the first failure. Run `cargo insta test` to run all snapshots.
test run::run_linked_environment_path ... FAILED

So.. the behavior is different on Linux than macOS?

@zanieb
Copy link
Member Author

zanieb commented Jan 29, 2025

Okay... so the test fails on Linux and macOS in CI so the behavior is different in CI than on my machine... that's annoying.

@zanieb
Copy link
Member Author

zanieb commented Jan 29, 2025

Another development — if I use Python provided by Homebrew, the link is resolved (e.g., VENV not TARGET) but if I use a uv-managed Python, it is not.

@zanieb
Copy link
Member Author

zanieb commented Jan 29, 2025

We do have non-snapshot tests for HomeBrew Python and the existing behavior is fine just not ideal, so I'm not too worried about shipping this regardless. It could be nice to understand why this does not change anything in that case, and if any downstream packagers will be affected by this change — but I think the worst-case scenario is that they need to skip the test.

@zanieb zanieb marked this pull request as ready for review January 29, 2025 21:51
@edmorley
Copy link
Contributor

edmorley commented Jan 29, 2025

Thank you for looking into this!

Another development — if I use Python provided by Homebrew, the link is resolved (e.g., VENV not TARGET) but if I use a uv-managed Python, it is not.

Reading the CPython implementation comments for executable/prefix/... calculation (since it's not covered in detail in the user-facing docs):
https://github.com/python/cpython/blob/a1a4e9f39ad86359e148fd193089b3b2a354f71d/Modules/getpath.py#L93-L153

...I see:

Before any searches are done, the location of the executable is determined. If Py_SetPath() was called, or if we are running on Windows, the 'real_executable' path is used (if known). Otherwise, we use the config-specified program name or default to argv[0].

I'm not sure what "config-specified" means - could that be _sysconfigdata__* - and thus depend on whether the install has been relocated/whether that metadata references a real path and otherwise will be ignored? The CPython implementation after the comments linked above also has quite a few darwin-specific conditionals. (I only skimmed that file in a browser and it's 800 lines long so these are mostly only lazy guesses while I'm watching TV - sorry! 😆 )

Copy link
Member

@charliermarsh charliermarsh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no need to absolutize this, right? It doesn't get returned to the user in any way -- just the sys.executable?

@zanieb
Copy link
Member Author

zanieb commented Jan 30, 2025

I don't think so. The only change would be in logs, I think. I can do an audit though.

@zanieb
Copy link
Member Author

zanieb commented Jan 30, 2025

Oh, we return root: interpreter.sys_prefix().to_path_buf(), so.. yeah there's no need to absolutize the path.

@zanieb zanieb merged commit d281f49 into main Jan 30, 2025
73 checks passed
@zanieb zanieb deleted the zb/fix-canonical branch January 30, 2025 16:10
tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request Feb 4, 2025
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.5.25` -> `0.5.27` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>astral-sh/uv (astral-sh/uv)</summary>

### [`v0.5.27`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0527)

[Compare Source](astral-sh/uv@0.5.26...0.5.27)

##### Enhancements

-   Avoid setting permissions during tar extraction ([#&#8203;11191](astral-sh/uv#11191))
-   Remove warnings for missing lower bounds ([#&#8203;11195](astral-sh/uv#11195))
-   Update PubGrub to set-based outdated priority tracking ([#&#8203;11169](astral-sh/uv#11169))
-   Improve error messages for `uv pip install` with `--extra` or `--all-extras` and invalid sources ([#&#8203;11193](astral-sh/uv#11193))
-   Sign Docker images using GitHub attestations ([#&#8203;8685](astral-sh/uv#8685))

##### Preview features

-   Don't expand self-referential extras in the build backend ([#&#8203;11142](astral-sh/uv#11142))

##### Performance

-   Filter discovered Python executables by source before querying ([#&#8203;11143](astral-sh/uv#11143))
-   Optimize exclusion computation for markers ([#&#8203;11158](astral-sh/uv#11158))
-   Use Astral-maintained `tokio-tar` fork ([#&#8203;11174](astral-sh/uv#11174))
-   Remove unneeded `.clone()` ([#&#8203;11127](astral-sh/uv#11127))

##### Bug fixes

-   Fix relative paths in bytecode compilation ([#&#8203;11177](astral-sh/uv#11177))
-   Percent-decode URLs in canonical comparisons ([#&#8203;11088](astral-sh/uv#11088))
-   Respect concurrency limits in parallel index fetch ([#&#8203;11182](astral-sh/uv#11182))
-   Use wire JSON schema for conflict items ([#&#8203;11196](astral-sh/uv#11196))
-   Use explicit `_GLibCVersion` tuple in uv-python crate ([#&#8203;11122](astral-sh/uv#11122))

##### Documentation

-   Add Git SHA locking behavior to docs ([#&#8203;11125](astral-sh/uv#11125))
-   Add best-practice flags to `pip install` example in troubleshooting guide ([#&#8203;11194](astral-sh/uv#11194))
-   Set `VIRTUAL_ENV` in Jupyter kernels ([#&#8203;11155](astral-sh/uv#11155))
-   Add instructions for deactivating an environment ([#&#8203;11200](astral-sh/uv#11200))

### [`v0.5.26`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0526)

[Compare Source](astral-sh/uv@0.5.25...0.5.26)

##### Enhancements

-   Add support for `uvx python` ([#&#8203;11076](astral-sh/uv#11076))
-   Allow `--no-dev --invert` in `uv tree` ([#&#8203;11068](astral-sh/uv#11068))
-   Update `uv python install --reinstall` to reinstall all previous versions ([#&#8203;11072](astral-sh/uv#11072))
-   Consistently write log messages with capitalized first word ([#&#8203;11111](astral-sh/uv#11111))
-   Suggest `--build-backend` when `--backend` is passed to `uv init` ([#&#8203;10958](astral-sh/uv#10958))
-   Improve retry trace message ([#&#8203;11108](astral-sh/uv#11108))

##### Performance

-   Remove unnecessary UTF-8 conversion in hash parsing ([#&#8203;11110](astral-sh/uv#11110))

##### Bug fixes

-   Ignore non-hash fragments in HTML API responses ([#&#8203;11107](astral-sh/uv#11107))
-   Avoid resolving symbolic links when querying Python interpreters ([#&#8203;11083](astral-sh/uv#11083))
-   Avoid sharing state between universal and non-universal resolves ([#&#8203;11051](astral-sh/uv#11051))
-   Error when `--script` is passing a non-PEP 723 script ([#&#8203;11118](astral-sh/uv#11118))
-   Make metadata deserialization failures non-fatal in the cache ([#&#8203;11105](astral-sh/uv#11105))
-   Mark metadata as dynamic when reading from built wheel cache ([#&#8203;11046](astral-sh/uv#11046))
-   Propagate credentials for `<index>/simple` to `<index>/...` endpoints ([#&#8203;11074](astral-sh/uv#11074))
-   Fix conflicting extra bug during `uv sync` ([#&#8203;11075](astral-sh/uv#11075))

##### Documentation

-   Add PyTorch XPU instructions to the PyTorch guide ([#&#8203;11109](astral-sh/uv#11109))
-   Add docs for signal handling ([#&#8203;11041](astral-sh/uv#11041))
-   Explain build frontend vs. build backend ([#&#8203;11094](astral-sh/uv#11094))
-   Fix formatting of `RUST_LOG` documentation ([#&#8203;10053](astral-sh/uv#10053))
-   Fix typo in `--no-deps` description ([#&#8203;11073](astral-sh/uv#11073))
-   Reflow CLI documentation comments ([#&#8203;11040](astral-sh/uv#11040))
-   Shorten "Using existing Python versions" nav item so it fits on one line ([#&#8203;11077](astral-sh/uv#11077))
-   Some minor touch-ups to the Python install guide ([#&#8203;11116](astral-sh/uv#11116))
-   Update Dependabot tracking issue link ([#&#8203;11054](astral-sh/uv#11054))
-   Update documentation for running in a container ([#&#8203;11052](astral-sh/uv#11052))
-   Upgrade PyTorch version in documentation ([#&#8203;11114](astral-sh/uv#11114))
-   Use `sys_platform` in lieu of `platform_system` in PyTorch docs ([#&#8203;11113](astral-sh/uv#11113))
-   Use positive (rather than negative) markers in PyTorch examples ([#&#8203;11112](astral-sh/uv#11112))
-   Fix unnecessary backslashes in brackets ([#&#8203;11059](astral-sh/uv#11059))
-   Suggest setting copy link mode in GitLab integration guide ([#&#8203;11067](astral-sh/uv#11067))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xNDMuMCIsInVwZGF0ZWRJblZlciI6IjM5LjE1Ni4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Entrypoint script shebang uses resolved path for symlinked Python rather than sys.executable
3 participants