Preliminary support for Pixi projects using pixi.toml #456
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jherland
force-pushed
the
jherland/pixi-toml-support
branch
2 times, most recently
from
c3e8f1e
to
2de6d56
Compare
jherland
force-pushed
the
jherland/pixi-toml-support
branch
9 times, most recently
from
d76d45e
to
8198e99
Compare
For a Python environment to be considered valid, we typically look for a Python executable, and an associated site-packages directory. The check for the Python executable uses .is_file(), which returns true as long the Python executable is a regular file, or a symlink to an existing regular file. We are however running into occasional CI failures where the Python executables in some of our (cached) virtualenvs for real_projects appear to be broken symlinks. I suspect that environments where Python executables are broken symlinks might also occur in other circumstances (e.g. on Nix where the underlying Python executable might have been garbage-collected due to a missing GC root, or on a more regular Linux distro where a stale virtualenv might be pointing to a system Python that has since been upgraded). In any case, a Python environment with a broken python symlink might be non-functional, but should that really cause problems for FawltyDeps? Since FawltyDeps can traverse the environment without relying on the environment's Python executable, the answer is actually no: All we really need is for bin/python to serve as a marker that this is (or at some point was) a Python environment, and a broken symlink should not stop us. Hence tweak our code to take broken symlinks into account when recognizing Python environments.
This adds support for the default configuration file used in Pixi project: pixi.toml. The relevant fields in this file mirror the Pixi fields we already parse in pyproject.toml (but without the tool.pixi prefix). Thus we can reuse the parsing functions we already wrote for supporting Pixi projects via pyproject.toml.
Now that we can parse dependency declarations from pixi.toml files, we need to expose this functionality in our docs + CLI, as well as automatically find pixi.toml files while traversing projects.
This was created with the following commands: - pixi init pixi_default_example - cd pixi_default_example/ - pixi add python - pixi add --pypi requests - pixi add ninja - echo "import requests" > main.py followed by removing/truncating files that I do not currently consider to be important for FawltyDeps to make sense of this project. The result is a simple Pixi project with two declared Conda dependencies (Python itself + ninja) and one PyPI dependency (requests), along with a single source file that imports requests. The expected result of running FawltyDeps here is reporting ninja as an unused dependency. This is because ninja is a Conda package with no Python components, and since it is not found as a Python package, we fall back to the identity mapping and expect it to be imported as "ninja". We should fix this by digging into the Conda environment to discover that ninja has no Python components, and therefore is not expected to be imported.
When parsing a pyproject.toml or pixi.toml with invalid TOML data, the tomllib parser will raise an exception that is not properly caught or handled by FawltyDeps. Fix this by logging an error message and returning from the parse function. This allows FawltyDeps to continue parsing other files and reporting undeclared/unused dependencies, whereas before it would abort with a traceback.
jherland
force-pushed
the
jherland/pixi-toml-support
branch
from
8198e99
to
f773397
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Parsing
pixi.tomlis not so different from parsing Pixi fields insidepyproject.toml. This adds support for Pixi projects that use thepixi.tomlconfiguration file.NOTE: The same Conda caveat still applies: We do not currently differentiate between Conda dependencies and PyPI dependencies, meaning that we assume that a Conda dependency named FOO will map one-to-one to a Python package named FOO. This is certainly not true for Conda dependencies that are not Python packages, and it probably isn't even true for all Conda dependencies that do indeed include Python packages.
Commits:
PyEnvSource: Slightly loosen restrictions on Python environmentsextract_declared_dependencies.py: Add support for parsingpixi.tomlpixi.tomlsupport to the rest of FawltyDepssample_projects: Add Pixi project w/default settingstest_traverse_project: Add tests using the Pixi sample projectsextract_declared_dependencies: Handle exceptions due to invalid toml