`virtualenv_starter()` does not find python locations with an unusual python version

[lschneiderbauer]

The way virtualenv_starter() is currently implemented, python locations with python versions that include characters at the end are discarded.
E.g. a python version with

$ python -EV
3.10.16+PRE

or similar is not recognized.

The reason is probably the use of numeric_version() in the find_starters() function inside virtualenv_starter().

I stumbled across this issue because my company appends a company-branding tag to the python version, and reticulate has problems detecting my python installations.

If you insist on having numeric version, would it be a reasonable solution to trim all the characters in the end before trying to convert to a numeric version?

[t-kalinowski]

Thanks for reporting.

We could do remove all characters after "+", with something like:

ver <- "3.10.16+PRE"
ver <- sub("\\+.*$", "", ver)
numeric_version(ver, FALSE)

However, the bigger issue is the missing Python prefix from the output. The standard output typically looks like this:

$ python3 -EV
Python 3.13.1

[lschneiderbauer]

However, the bigger issue is the missing Python prefix from the output.

That was actually a copy&paste mistake, sorry. The actual output is

$ python3 -EV
Python 3.10.16+PRE

[lschneiderbauer]

Is there some workaround I can apply until the fix is released?
My problem is that I can't get sparklyr::spark_connect() to work with databricks, because it wants to create its own environment, but does not find any starter (I think because of this bug). I also cannot pass any argument to tell sparklyr it should use my own environment.

[t-kalinowski]

You can install the dev branch with the patch like this:

remotes::install_github("rstudio/reticulate@relax-venv-starter-version-check")

Also, if you create a Python environment through other means, I believe sparklyr respects the RETICULATE_PYTHON env var if it is set. @edgararuiz is that correct?

[edgararuiz]

I don't quiet remember how we arrived at this one, but I recall that after some back and forth, we decided to disable it if set, unless you're inside a Connect server.

You should be able to pass envname to force the use of a given environment: https://spark.posit.co/deployment/databricks-connect.html#restricted-python-environments