Running tox from two processes concurrently is not threadsafe

[jberg7]

Issue

Running tox commands like tox run -e lint from two processes simultaneously often encounters a race condition when building the sdist. Each process uses the same temp dir at the project root, copies source files into this dir, tars it, and then deletes it. If one process deletes the temp dir before the other finishes, the other process will fail with a "No such file or directory" error.

Technically this is a problem with python -m build --sdist itself, but can be solved by specifying a unqiue output directory for each process using the --outdir option. However, as far as I can tell tox does not allow me to configure an --outdir for sdist. If such an option exists let me know! :)

Environment

Provide at least:

• OS: Ubuntu 22.04.5 LTS

Output of pip list of the host Python, where tox is installed

build                                  1.2.2.post1
setuptools                             75.6.0
tox                                    4.23.2

Output of running tox

Output of tox -rvv

sh test_build.sh 
Testing concurrent builds...
Starting build for process 1 in directory build_process_1...
Starting build for process 2 in directory build_process_2...
ROOT: 108 D setup logging to DEBUG on pid 123 [tox/report.py:222]
ROOT: 108 D setup logging to DEBUG on pid 456 [tox/report.py:222]
ROOT: 161 W will run in automatically provisioned tox, host /path/to/MyProject/venv/bin/python3.12 is missing [requires (has)]: tox-uv>=1.16.0 [tox/provision.py:125]
ROOT: 161 W will run in automatically provisioned tox, host /path/to/MyProject/venv/bin/python3.12 is missing [requires (has)]: tox-uv>=1.16.0 [tox/provision.py:125]
ROOT: 177 I find interpreter for spec PythonSpec(path=/path/to/MyProject/venv/bin/python3.12) [virtualenv/discovery/builtin.py:73]
ROOT: 177 D filesystem is case-sensitive [virtualenv/info.py:25]
ROOT: 178 D got python info of %s from (PosixPath('/usr/bin/python3.12'), PosixPath('/some/path/.local/share/virtualenv/py_info/1/some_hash.json')) [virtualenv/app_data/via_disk_folder.py:131]
ROOT: 178 I proposed PythonInfo(spec=CPython3.12.7.final.0-64, system=/usr/bin/python3.12, exe=/path/to/MyProject/venv/bin/python3.12, platform=linux, version='3.12.7 (main, Oct  1 2024, 08:52:12) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:80]
ROOT: 179 D accepted PythonInfo(spec=CPython3.12.7.final.0-64, system=/usr/bin/python3.12, exe=/path/to/MyProject/venv/bin/python3.12, platform=linux, version='3.12.7 (main, Oct  1 2024, 08:52:12) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:82]
ROOT: 179 I find interpreter for spec PythonSpec(path=/path/to/MyProject/venv/bin/python3.12) [virtualenv/discovery/builtin.py:73]
ROOT: 180 D filesystem is case-sensitive [virtualenv/info.py:25]
ROOT: 181 D got python info of %s from (PosixPath('/usr/bin/python3.12'), PosixPath('/some/path/.local/share/virtualenv/py_info/1/some_hash.json')) [virtualenv/app_data/via_disk_folder.py:131]
ROOT: 181 I proposed PythonInfo(spec=CPython3.12.7.final.0-64, system=/usr/bin/python3.12, exe=/path/to/MyProject/venv/bin/python3.12, platform=linux, version='3.12.7 (main, Oct  1 2024, 08:52:12) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:80]
ROOT: 181 D accepted PythonInfo(spec=CPython3.12.7.final.0-64, system=/usr/bin/python3.12, exe=/path/to/MyProject/venv/bin/python3.12, platform=linux, version='3.12.7 (main, Oct  1 2024, 08:52:12) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:82]
ROOT: 200 I will run in a automatically provisioned python environment under /path/to/MyProject/build_process_2/.tox/bin/python [tox/provision.py:146]
ROOT: 202 W provision> build_process_2/.tox/bin/python -m tox run -e lint --workdir build_process_2 -vv [tox/tox_env/api.py:427]
ROOT: 203 I will run in a automatically provisioned python environment under /path/to/MyProject/build_process_1/.tox/bin/python [tox/provision.py:146]
ROOT: 204 W provision> build_process_1/.tox/bin/python -m tox run -e lint --workdir build_process_1 -vv [tox/tox_env/api.py:427]
ROOT: 107 D setup logging to DEBUG on pid 2383474 [tox/report.py:222]
ROOT: 108 D setup logging to DEBUG on pid 2383479 [tox/report.py:222]
.pkg: 368 W _optional_hooks> python /path/to/MyProject/build_process_2/.tox/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta [tox/tox_env/api.py:427]
.pkg: 371 W _optional_hooks> python /path/to/MyProject/build_process_1/.tox/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta [tox/tox_env/api.py:427]
Backend: run command _optional_hooks with args {}
Backend: Wrote response {'return': {'get_requires_for_build_sdist': True, 'prepare_metadata_for_build_wheel': True, 'get_requires_for_build_wheel': True, 'build_editable': True, 'get_requires_for_build_editable': True, 'prepare_metadata_for_build_editable': True}} to /tmp/pep517__optional_hooks-iwfj3e3_.json
.pkg: 490 I exit None (0.12 seconds) /path/to/MyProject> python /path/to/MyProject/build_process_1/.tox/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=2383501 [tox/execute/api.py:286]
Backend: run command _optional_hooks with args {}
Backend: Wrote response {'return': {'get_requires_for_build_sdist': True, 'prepare_metadata_for_build_wheel': True, 'get_requires_for_build_wheel': True, 'build_editable': True, 'get_requires_for_build_editable': True, 'prepare_metadata_for_build_editable': True}} to /tmp/pep517__optional_hooks-2rz6np7u.json
.pkg: 495 I exit None (0.12 seconds) /path/to/MyProject> python /path/to/MyProject/build_process_2/.tox/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=2383495 [tox/execute/api.py:286]
.pkg: 532 W get_requires_for_build_sdist> python /path/to/MyProject/build_process_1/.tox/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta [tox/tox_env/api.py:427]
.pkg: 538 W get_requires_for_build_sdist> python /path/to/MyProject/build_process_2/.tox/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta [tox/tox_env/api.py:427]
Backend: run command get_requires_for_build_sdist with args {'config_settings': None}
Backend: run command get_requires_for_build_sdist with args {'config_settings': None}
running egg_info
writing MyProject.egg-info/PKG-INFO
writing dependency_links to MyProject.egg-info/dependency_links.txt
writing requirements to MyProject.egg-info/requires.txt
writing top-level names to MyProject.egg-info/top_level.txt
running egg_info
writing MyProject.egg-info/PKG-INFO
writing dependency_links to MyProject.egg-info/dependency_links.txt
writing requirements to MyProject.egg-info/requires.txt
writing top-level names to MyProject.egg-info/top_level.txt
reading manifest file 'MyProject.egg-info/SOURCES.txt'
writing manifest file 'MyProject.egg-info/SOURCES.txt'
Backend: Wrote response {'return': []} to /tmp/pep517_get_requires_for_build_sdist-c7br0rnt.json
.pkg: 881 I exit None (0.35 seconds) /path/to/MyProject> python /path/to/MyProject/build_process_1/.tox/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=2383517 [tox/execute/api.py:286]
reading manifest file 'MyProject.egg-info/SOURCES.txt'
writing manifest file 'MyProject.egg-info/SOURCES.txt'
Backend: Wrote response {'return': []} to /tmp/pep517_get_requires_for_build_sdist-s6mr8x84.json
.pkg: 889 I exit None (0.35 seconds) /path/to/MyProject> python /path/to/MyProject/build_process_2/.tox/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=2383523 [tox/execute/api.py:286]
.pkg: 924 W build_sdist> python /path/to/MyProject/build_process_1/.tox/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta [tox/tox_env/api.py:427]
.pkg: 931 W build_sdist> python /path/to/MyProject/build_process_2/.tox/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta [tox/tox_env/api.py:427]
Backend: run command build_sdist with args {'sdist_directory': '/path/to/MyProject/build_process_1/.pkg/dist', 'config_settings': None}
Backend: run command build_sdist with args {'sdist_directory': '/path/to/MyProject/build_process_2/.pkg/dist', 'config_settings': None}
running sdist
running egg_info
writing MyProject.egg-info/PKG-INFO
writing dependency_links to MyProject.egg-info/dependency_links.txt
writing requirements to MyProject.egg-info/requires.txt
writing top-level names to MyProject.egg-info/top_level.txt
running sdist
running egg_info
writing MyProject.egg-info/PKG-INFO
writing dependency_links to MyProject.egg-info/dependency_links.txt
writing requirements to MyProject.egg-info/requires.txt
writing top-level names to MyProject.egg-info/top_level.txt
reading manifest file 'MyProject.egg-info/SOURCES.txt'
writing manifest file 'MyProject.egg-info/SOURCES.txt'
warning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md

running check
creating MyProject-0.0.1
creating MyProject-0.0.1/sub_project
...
copying files to MyProject-0.0.1...
copying pyproject.toml -> MyProject-0.0.1
copying sub_project/foobar.py -> MyProject-0.0.1/sub_project
...
Writing MyProject-0.0.1/setup.cfg
Creating tar archive
copying files to MyProject-0.0.1...
copying pyproject.toml -> MyProject-0.0.1
copying sub_project/foobar.py -> MyProject-0.0.1/sub_project
...
Writing MyProject-0.0.1/setup.cfg
Creating tar archive
removing 'MyProject-0.0.1' (and everything under it)
Traceback (most recent call last):
  File "/path/to/MyProject/build_process_2/.pkg/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
    dist.run_commands()
  File "/path/to/MyProject/build_process_2/.pkg/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 983, in run_commands
    self.run_command(cmd)
  File "/path/to/MyProject/build_process_2/.pkg/lib/python3.12/site-packages/setuptools/dist.py", line 999, in run_command
    super().run_command(command)
  File "/path/to/MyProject/build_process_2/.pkg/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1002, in run_command
    cmd_obj.run()
  File "/path/to/MyProject/build_process_2/.pkg/lib/python3.12/site-packages/setuptools/command/sdist.py", line 69, in run
    self.make_distribution()
Backend: Wrote response {'code': "error: [Errno 2] No such file or directory: 'MyProject-0.0.1/sub_project/__init__.py'", 'exc_type': 'SystemExit', 'exc_msg': "error: [Errno 2] No such file or directory: 'MyProject-0.0.1/sub_project/__init__.py'"} to /tmp/pep517_build_sdist-63hmpvb6.json

Minimal example

#!/bin/bash

# Script to test concurrent builds with unique output directories

build_library() {
  local process_id=$1
  local outdir="build_process_$process_id"
  local subdir="dir_$process_id"  

  echo "Starting build for process $process_id in directory $outdir..."
  
  # python -m build --sdist --outdir=$outdir
  tox run -e lint --workdir $outdir -vv

  if [ $? -eq 0 ]; then
    echo "Build for process $process_id completed successfully."
  else
    echo "Build for process $process_id failed."
  fi
}

rm -rf dist/

echo "Testing concurrent builds..."
build_library 1 &
build_library 2 &

wait

echo "All builds complete."

[jberg7]

Hopefully there's an existing configuration for this that I just don't know about. But if not, I can think of two possible solutions:

• The tox --workdir command should be used somehow for the sdist --outdir
• Allow users to configure the sdist outdir directly from the tox command line

I can work around the race condition slightly by using the --skip-pkg-install cli option, but I don't really want to just disable the build altogether. I have also seen tox run-parallel, but for my use case I need multiple distinct processes.