dev not picking up comments in pyproject.toml

[j1philli]

After refreshing an environment, the pkgx comments nor python declaration in the file are enforcing the python version in the dev environment.

Discussed in https://github.com/orgs/pkgxdev/discussions/768

^{Originally posted by j1philli October 2, 2023}
HI!
Just switched from tea to pkgx and went to get dev setup.

Because of issues with python 3.12 and my project I want to have it use 3.11.4 so I put it in the pyproject.toml

But alas I still have 3.12 in my dev environment

Any guidance would be much appreciated!

[mxcl]

did this use to work pre 1.0?

[mxcl]

k def not, will fix for 1.0.1

[mxcl]

hmm k, first I need some docs about how this python spec is meant to work. Who would normally make the python this version? And browsing poetry docs I see that the python dep can be specified in many places. What should we do in such a case? Add all the pythons to the env? We can do that just need clarification.

[j1philli]

Who would normally make the python this version?
Sorry not sure I understand the question, isn't pinning versions of software pretty common place?

What should we do in such a case?
Installing all versions would not work. I wouldn't try and pull the version from the dependency list pyproject.toml because of the issue you specified. I would just make the pkgx comments work so that the user is able to specify the version they need in environment. I think if there are no comments then installing latest version like it does now is good enough.

I am surprised that you can't just send a command while in dev to change the version of an installed dependency

[mxcl]

Sorry not sure I understand the question, isn't pinning versions of software pretty common place?

who enforces the python version, who installs it? the python package manager itself? The user?

I am surprised that you can't just send a command while in dev to change the version of an installed dependency

We can do this. I'm just trying to understand what the python community has decided is correct behavior so pkgx can do it.

[j1philli]

who enforces the python version, who installs it? the python package manager itself? The user?

The user most of the time, then there are python utilities that also manage its own, seperate versions in environments they create(I don't think pkgx needs to worry about the latter at all).
The user needs to have python on machine. This is what pkgx should handle similar to the way https://github.com/pyenv/pyenv where you can specify different versions to use.
Then there are python environment managers like https://github.com/python-poetry/poetry that will use the systems python to run and then it manages environments with different versions of python and python packages installed in the environments it creats. Again, I don't think pkgx should be worried about these.

This has only come up because the poetry python dependency in the pantry wasn't specific enough to not use 3.12, so I wanted to override it somehow. I think the comments should override it if the specified version in the override is within dependency restraints. If its not get a confirmation from the user.
Once pkgx gets the latest pantry version I believe it will install the proper version in dev and everything will work.

Hope that builds context, I would still consider myself a novice so let me know if something isn't clear.

[michjnich]

Python person here - when it comes to the management of the version of python itself for a project, different tools may specify it in different ways in config, but all of them expect a matching version to be installed already for it to work. If there isn't a version that matches, it won't work.

So, if you for example only have python 3.11.4 on your machine:

• Specifying 3.11.4 will work
• Specifying 3.11 or ~3.11.x will or even ^3.x.y work and give you 3.11.4 (wouldn't do that last one though :) )
• Specifying 3.11.6 will not work becuase there's only one version that matches and it's not on your machine.

Installation of different python versions is often (not always) handled via another tool (eg. asdf, pyenv etc.)

[mxcl]

Tests cannot confirm the first part of this bug report, a pyproject.toml that has:

# pyproject. toml
# ---
# pkgx:
#   python@3.10
# ---

Results in python 3.10. Can you provide a test case?

The second part is a feature request to load python versions from the TOML which will go into 1.1

[j1philli]

Not the results I am seeing. I have a pyproject.toml with only what you stated as comments. I enter the directory it is located which is brand new. I check the version of python and it is the latest 3.11.6 instead of the expected 3.10

[mxcl]

k figured it out, fix coming up thanks. This only effects YAML FM in pyproject.toml that was a string with a pkgx: prefix.