Github Actions running reticulate with Python 3.8.12 fails to create venv

[pbulsink]

Reticulate in Github Actions seems to be failing to create a venv when Python 3.12.8 is used.

Using the prescribed Github Action for setup r-reticulate produces the following error:

      - name: setup r-reticulate venv
        shell: Rscript {0}
        run: |
              python_packages <- c("numpy", "fastf1")
              
              library(reticulate)
              virtualenv_create("r-reticulate", Sys.which("python"))
              virtualenv_install("r-reticulate", python_packages)
              
              path_to_python <- virtualenv_python("r-reticulate")
              writeLines(sprintf("RETICULATE_PYTHON=%s", path_to_python),
                         Sys.getenv("GITHUB_ENV"))

output:

  shell: /usr/local/bin/Rscript {0}
  env:
    GITHUB_PAT: ***
    R_KEEP_PKG_SOURCE: yes
    R_LIBS_USER: /home/runner/work/_temp/Library
    TZ: UTC
    _R_CHECK_SYSTEM_CLOCK_: FALSE
    NOT_CRAN: true
    RSPM: https://packagemanager.posit.co/cran/__linux__/noble/latest
    RENV_CONFIG_REPOS_OVERRIDE: https://packagemanager.posit.co/cran/__linux__/noble/latest
    pythonLocation: /opt/hostedtoolcache/Python/3.12.8/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.12.8/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.8/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.8/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.8/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.12.8/x64/lib
    R_LIB_FOR_PAK: /opt/R/4.4.2/lib/R/site-library
Error in stop_no_virtualenv_starter(version = version, python = python) : 
  Suitable Python installation for creating a venv not found.
  Requested Python: /opt/hostedtoolcache/Python/3.12.8/x64/bin/python
Please install Python with one of following methods:
- https://github.com/rstudio/python-builds/
- reticulate::install_python(version = '<version>')
Calls: virtualenv_create ... check_can_be_virtualenv_starter -> stop_no_virtualenv_starter
Execution halted

First noticed in Scasanova/f1dataR weekly checks: https://github.com/SCasanova/f1dataR/actions/runs/12344927770/job/34448193501. Previous weekly check passed, no code was checked in during the week. The only change was the release of Python 3.12.8 and use of that version by actions/setup-python@v5.

Also visible in reticulate's own R-CMD-Check test actions: https://github.com/rstudio/reticulate/actions/runs/12770139964/job/35594285240.

I apologize, I don't have a device on which I can test this locally.

[t-kalinowski]

Thanks for reporting, this is fixed in the development version now.

The issue was that the 'setup-python' Github action, for Python 3.12, also set the env var LD_LIBRARY_PYTHON to /opt/hostedtoolcache/Python/3.12.8/x64/lib.

R's startup routine also modified the LD_LIBRARY_PATH, so that in the R session the final LD_LIBRARY_PATH entries were:

/opt/R/4.4.2/lib/R/lib
/usr/local/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/jvm/temurin-17-jdk-amd64/lib/server
/opt/hostedtoolcache/Python/3.12.8/x64/lib

This in turned caused the wrong libpython to be loaded when Python was called from R.
After sshing into the runner, this is what I saw:

runner@fv-az1118-779:~/work/gh-actions-playground/gh-actions-playground$ ldd $(which python)
        linux-vdso.so.1 (0x00007ffdaa74e000)
        libpython3.12.so.1.0 => /opt/hostedtoolcache/Python/3.12.8/x64/lib/libpython3.12.so.1.0 (0x00007f6a96600000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6a96200000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f6a96c91000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f6a96d8b000)
runner@fv-az1118-779:~/work/gh-actions-playground/gh-actions-playground$ R -q -e 'system("ldd $(which python)")'
> system("ldd $(which python)")
        linux-vdso.so.1 (0x00007ffeb6b93000)
        libpython3.12.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython3.12.so.1.0 (0x00007f3b6bc00000)
        libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6 (0x00007f3b6b800000)
        libm.so.6 => /usr/lib/x86_64-linux-gnu/libm.so.6 (0x00007f3b6c4b8000)
        libz.so.1 => /usr/lib/x86_64-linux-gnu/libz.so.1 (0x00007f3b6bbe4000)
        libexpat.so.1 => /usr/lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f3b6bbb9000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f3b6c5a8000)
> 
> 
runner@fv-az1118-779:~/work/gh-actions-playground/gh-actions-playground$ R -q -e 'system("LD_LIBRARY_PATH=/opt/hostedtoolcache/Python/3.12.8/x64/lib ldd $(which python)")'                                                                                                                                                           
> system("LD_LIBRARY_PATH=/opt/hostedtoolcache/Python/3.12.8/x64/lib ldd $(which python)")
        linux-vdso.so.1 (0x00007ffcf272c000)
        libpython3.12.so.1.0 => /opt/hostedtoolcache/Python/3.12.8/x64/lib/libpython3.12.so.1.0 (0x00007f55f4800000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f55f4400000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f55f4717000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f55f4eeb000)
> 

Note that the key difference:

libpython3.12.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython3.12.so.1.0
# vs 
libpython3.12.so.1.0 => /opt/hostedtoolcache/Python/3.12.8/x64/lib/libpython3.12.so.1.0 

This also explains the discrepancy in the reported Python version I noticed early on (3.12.8 vs 3.12.3):

> system("python -m pip")
/opt/hostedtoolcache/Python/3.12.8/x64/bin/python: No module named pip
> system("python -V")
Python 3.12.3
> system("LD_LIBRARY_PATH=/opt/hostedtoolcache/Python/3.12.8/x64/lib python -V")
Python 3.12.8
> system("LD_LIBRARY_PATH=/opt/hostedtoolcache/Python/3.12.8/x64/lib python -m pip")

Usage:   
  /opt/hostedtoolcache/Python/3.12.8/x64/bin/python -m pip <command> [options]

< omitted pip --help output >

This also explains the discrepency between the initialized sys.path in and out of the R session.

> system2(python, c("-m", "site"))                                                                                                                                                                                                                                                            
sys.path = [
    '/home/runner/work/gh-actions-playground/gh-actions-playground',
    '/opt/hostedtoolcache/Python/3.12.8/x64/lib/python312.zip',
    '/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12',
    '/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12/lib-dynload',
]
USER_BASE: '/home/runner/.local' (exists)
USER_SITE: '/home/runner/.local/lib/python3.12/site-packages' (doesn't exist)
ENABLE_USER_SITE: True
> 
> 

Save workspace image? [y/n/c]: y

runner@fv-az1672-703:~/work/gh-actions-playground/gh-actions-playground$ /opt/hostedtoolcache/Python/3.12.8/x64/bin/python3.12 -m site
sys.path = [
    '/home/runner/work/gh-actions-playground/gh-actions-playground',
    '/opt/hostedtoolcache/Python/3.12.8/x64/lib/python312.zip',
    '/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12',
    '/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12/lib-dynload',
    '/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12/site-packages',
]
USER_BASE: '/home/runner/.local' (exists)
USER_SITE: '/home/runner/.local/lib/python3.12/site-packages' (doesn't exist)
ENABLE_USER_SITE: True