Skip to content

Too many files after moving from v1.2.0 to v2.0.0 and v2.2.0 #527

Open
@jkleckner

Description

@jkleckner

In a large Scala repo with submodules, I tested a lot of combinations of sbt versions and sbt-assembly versions and found that moving from sbt-assembly version 1.2.0 to 2.0.0 or 2.2.0 resulted in the message when running sbt assembly:

Could not create file /Users/.../data/target/streams/test/_global/_global/compileOutputs/previousjava.io.IOException: Too many open files

As mentioned in #463 (comment), there is increased memory use, but that is expected with the amount of caching going on.

The sbt-assembly version 1.2.0 worked with all sbt versions up to and including 1.10.1
But moving to 2.0.0 started getting that error on my Mac (64GB Intel Sonomo 14.5).
As described here, running ulimit -n 10240 (the maximum on a Mac) allowed sbt assembly to complete.

However, sbt Test/assembly would not work and would give "too many open files".
This part of an upgrade process from Scala 2.11.12 and Java 8 so perhaps that is a contributing factor.
Any suggestions on what to try?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions