Skip to content

Too many open files #1551

Closed
Closed

Description

Checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

[project]
...
channels = ["conda-forge"]
platforms = ["linux-64", "linux-aarch64", "osx-arm64"]

[dependencies]
beautifulsoup4 = ">=4,<5"
chromadb = "<1"
coverage = ">=7.4.1,<7.5"
cyclonedx-bom = ">=3.11.7,<3.12"
datamodel-code-generator = ">=0.25.4,<0.26"
dateparser = ">=1.2.0,<1.3"
easyocr = ">=1.7.1,<1.8"
fastapi = ">=0.103.0,<0.104"
httpx = ">=0.26.0,<0.27"
ipython = ">=8.21.0,<8.22"
jupyter = ">=1.0.0,<1.1"
lxml = ">=5.1.0,<5.2"
openai = ">=1.35.3,<1.36"
openpyxl = ">=3.1.2,<3.2"
pandas = "2.*"
passlib = ">=1.7.4,<1.8"
pdbpp = ">=0.10.3,<0.11"
pdf2image = "1.16.*"
pixi-pycharm = ">=0.0.5,<0.1"
polars = ">=0.20.23,<0.21"
py-spy = ">=0.3.14,<0.4"
py7zr = ">=0.20.8,<0.21"
pydantic = ">=2.6.1,<2.7"
pyexcel = ">=0.7.0,<0.8"
pyexcel-xls = ">=0.6.2,<0.7"
pyexcel-xlsx = ">=0.6.0,<0.7"
pymongo = ">=4.6.1,<4.7"
pypdf = ">=4.0.2,<4.1"
pytesseract = ">=0.3.13,<0.4"
pytest = "<8" # https://github.com/TvoroG/pytest-lazy-fixture/issues/65
python = ">=3.12"
python-dotenv = ">=1.0.1,<1.1"
python-jose = ">=3.3.0,<3.4"
python-multipart = ">=0.0.7,<0.1"
sentence-transformers = ">=2.6.1,<2.7"
tabulate = ">=0.9.0,<0.10"
tesseract = ">=5.3.4,<5.4"
testcontainers = ">=4.4.0,<5"
thefuzz = ">=0.22.1,<0.23"
tiktoken = ">=0.6.0,<0.7"
transformers = ">=4.39.2,<4.40"
uvicorn = ">=0.27.0.post1,<0.27.1"
xmlschema = ">=3.2.0,<3.3"
xmltodict = ">=0.13.0,<0.14"
python-xxhash = ">=3.4.1,<3.5"
pyarrow = "<16" # pandas incompat
requests = "<2.32" # https://github.com/docker/docker-py/issues/3256
pytorch = "<2.3"
pytest-lazy-fixture = ">=0.6.3,<0.7"
loguru = ">=0.7.2,<0.8"
python-docx = ">=1.1.2,<1.2"
markdown = ">=3.6,<4"
pytest-xdist = ">=3.6.1,<3.7"
waitress = ">=3.0.0,<3.1"
pyright = ">=1.1.368,<1.2"

Issue description

Running a large new environment for the first time, on macOS:

❯ pixi run python
  ⠂ default:linux-aarch6400:00:03] resolving conda
  ⠂ default:linux-64     [00:00:03] loading repodata
    ⠙ validate cache       [00:00:00] [━━━━━━━━━━━━━━──────] tesseract (+167)
    ⠙ download & extract   [00:00:00] [────────────────────]      0 B @ 0 B/s    scikit-learn (+100)
    ⠙ installing packages  [00:00:00] [────────────────────]     0/536 sentencepiece-python (+161)                                               × failed to instantiate a prefix for 'default'
  ├─▶ failed to link xmltodict-0.13.0-pyhd8ed1ab_0.tar.bz2
  ├─▶ failed to read 'index.json'
  ╰─▶ Too many open files (os error 24)
❯ wc -l pixi.toml
     105 pixi.toml
❯ wc -l pixi.lock
   25153 pixi.lock
❯ grep platform pixi.toml
platforms = ["linux-64", "linux-aarch64", "osx-arm64"]

Expected behavior

Should do fewer things in parallel so as to not go over macOS ulimit -n

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

🐞 bugSomething isn't workingcondaIssue related to Conda dependenciesneeds-designNeeds a design so it can be implemented

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions