Pipenv installation of basic packages failing on Windows runners

[chrisjkuch]

See for instance this and #451 builds.

[chrisjkuch]

Something funky is going on.

The error message is a little unhelpful because I don't know what package is failing to resolve. By bypassing pipenv install and just trying to do pipenv run pip install -r requirements.txt, I end up finding that there's a supposed python architecture mismatch when installing matplotlib.

Project name: matplotlib
...
Need python for x86_64, but found x86

For some reason, within the makefile command make create_environment, the environment is being created using a 32 bit version of Python:

======================= STDERR ======================
++ basename 'C:\Users\runneradmin\AppData\Local\Temp\tmptoc68acjdata-project\my-test-repo-0'
+ PROJECT_NAME=my-test-repo-0
++ dirname 'D:\a\cookiecutter-data-science\cookiecutter-data-science\tests\pipenv_harness.sh'
+ CCDS_ROOT='D:\a\cookiecutter-data-science\cookiecutter-data-science\tests'
+ MODULE_NAME=project_module
+ trap finish EXIT
+ source 'D:\a\cookiecutter-data-science\cookiecutter-data-science\tests/test_functions.sh'
+ cd 'C:\Users\runneradmin\AppData\Local\Temp\tmptoc68acjdata-project\my-test-repo-0'
+ make
+ make create_environment
Loading .env environment variables...

Creating a virtualenv for this project

Pipfile: 

C:\Users\runneradmin\AppData\Local\Temp\tmptoc68acjdata-project\my-test-repo-0\

Pipfile

Using C:/hostedtoolcache/windows/Python/3.11.9/x86/python.exe3.11.9 to create 

virtualenv...

created virtual environment CPython3.11.9.final.0-32 in 3110ms

  creator 

[chrisjkuch]

Explicitly setting the python path when creating the virtual environment passes:

# in pipenv_harness.sh
# make create_environment
pipenv --python C:/hostedtoolcache/windows/Python/3.11.9/x64/python.exe

I wonder if we got hit with Pipenv's changes to pythonfinder, our first issue with this popped up on April 27th scheduled run (https://github.com/drivendataorg/cookiecutter-data-science/actions/runs/14686564792/job/41232512661#logs)

[chrisjkuch]

Okay confirmed that pinning pipenv<2025 leads to passing tests.

[jayqi]

Yeah, seems weird that it chooses

Using C:/hostedtoolcache/windows/Python/3.11.9/x86/python.exe3.11.9 to create 

It seems like the tool cache on Windows runners have both x86 and x64 installed, and maybe there's something about the ordering in PATH or something like that that makes it pick the x86 one.

It sounds like Pipenv doesn't specify the architecture in lockfiles. pypa/pipenv#2397

I don't think it supports cross-platform lockfiles either, even. pypa/pipenv#5130

[jayqi]

We have a bunch of layers:

• Invoking pytest
• The test running the test harness shell script
• The shell script running Make
• Make invoking Pipenv

where environment variables can be lost, so it may be that we should expect that Pipenv should see actions/setup-python's Python at the front of PATH but it got dropped somewhere.

[chrisjkuch]

It seems like the tool cache on Windows runners have both x86 and x64 installed, and maybe there's something about the ordering in PATH or something like that that makes it pick the x86 one.

That seems to be the case, though why exactly that's the case is not clear to me.

=== DEBUGGING PYTHON DISCOVERY ===
--- Using pythonfinder and listing versions ---
Found Python 3.13.post32: C:\hostedtoolcache\windows\Python\3.13.3\x86\python.exe (arch: None)
Found Python 3.13: C:\hostedtoolcache\windows\Python\3.13.3\x64\python.exe (arch: None)
Found Python 3.12.post32: C:\hostedtoolcache\windows\Python\3.12.10\x86\python.exe (arch: None)
Found Python 3.12: C:\hostedtoolcache\windows\Python\3.12.10\x64\python.exe (arch: None)
Found Python 3.11.9: C:\hostedtoolcache\windows\Python\3.11.9\x64\python.exe (arch: None)
Found Python 3.11.9: C:\hostedtoolcache\windows\Python\3.11.9\x64\python3.exe (arch: None)
Found Python 3.11.post32: C:\hostedtoolcache\windows\Python\3.11.9\x86\python.exe (arch: None)
Found Python 3.10.post32: C:\hostedtoolcache\windows\Python\3.10.11\x86\python.exe (arch: None)
Found Python 3.10: C:\hostedtoolcache\windows\Python\3.10.11\x64\python.exe (arch: None)
Found Python 3.9.13: C:\hostedtoolcache\windows\Python\3.9.13\x64\python.exe (arch: None)
Found Python 3.9.13: C:\hostedtoolcache\windows\Python\3.9.13\x64\python3.exe (arch: None)
Found Python 3.9.post32: C:\hostedtoolcache\windows\Python\3.9.13\x86\python.exe (arch: None)
Found Python 3.8.post32: C:\hostedtoolcache\windows\Python\3.8.10\x86\python.exe (arch: None)
Found Python 3.8: C:\hostedtoolcache\windows\Python\3.8.10\x64\python.exe (arch: None)

[jayqi]

@chrisjkuch

Found Python 3.13.post32: C:\hostedtoolcache\windows\Python\3.13.3\x86\python.exe (arch: None)

Found Python 3.12.post32: C:\hostedtoolcache\windows\Python\3.12.10\x86\python.exe (arch: None)

Found Python 3.11.post32: C:\hostedtoolcache\windows\Python\3.11.9\x86\python.exe (arch: None)
Found Python 3.10.post32: C:\hostedtoolcache\windows\Python\3.10.11\x86\python.exe (arch: None)

These are actually kind of suspicious. These clearly are printing out wrong versions — it should either read, e.g., 3.13.3.post32 or just 3.13.3.

What are you using to print these? Is this Pipenv's pythonfinder or some other pythonfinder?

[chrisjkuch]

I was using actual pythonfinder

# 1. Install pythonfinder and list all Python versions with paths
echo "--- Using pythonfinder and listing versions ---"
python -c "
import pythonfinder
finder = pythonfinder.Finder()
pythons = finder.find_all_python_versions()
for python in pythons:
    print(f'Found Python {python.version}: {python.path} (arch: {python.architecture})')
"

[jayqi]

Okay, based on my understanding here, the pythonfinder package and Pipenv's pythonfinder are the same. It's vendored in Pipenv but the source code is the same as the standalone pythonfinder package.

pypa/pipenv#6360 (comment)

python.version does not seem to me like it's printing out a correct version identifier. I think it may be worth taking the time to try to do a more minimal repro to (1) confirm pythonfinder has a bug and (2) this bug affects Pipenv's Python discovery. I think the way to do this is probably to make a separate Windows GitHub Actions workflow that just tries to isolate the behavior of pythonfinder and Pipenv (probably not in this repository — I do stuff like this in jayqi/github-actions-sandbox). Up to you if you want to do it or be involved, I'll try to get to it sometime.

I don't think this needs to block the quick fix you have in #453. We can just merge that.