pipe(): Too many open files

[braddunbar]

When running coffee on a directory with a large number of files (329 to be exact), I receive the following error:

pipe(): Too many open files

child_process.js:243
  var fds = this._internal.spawn(path,
                       ^
Error: Error spawning
    at ChildProcess.spawn (child_process.js:243:28)
    at child_process.js:31:15
    at child_process.js:77:15
    at child_process.js:38:27
    at /Users/brad/dev/coffee-script/lib/command.js:242:16
    at path.js:292:19

The failure seems to center on the call to mkdir -p. I've yet to find a way to fix this cleanly.

[braddunbar]

After several runs, the issue manifests as

EMFILE, Too many open files '/path/to/file'

Also, this was observed on OSX.

[lancejpollard]

I'm getting a similar error because of too many files open:

fs.js:632
    throw errnoException(errno, 'watch');
10:22:36 - compiled src/design.io/support.coffee
^
Error: watch EMFILE
    at errnoException (fs.js:605:11)
    at FSWatcher.start (fs.js:632:11)
    at Object.watch (fs.js:660:11)
    at Object.oncomplete (/Users/viatropos/Documents/git/personal/plugins/design.io/node_modules/coffee-script/lib/coffee-script/command.js:201:17)

This should just be written with something like the async.js library:

https://github.com/jashkenas/coffee-script/blob/master/src/command.coffee#L106

[Crash--]

I'm getting the same error that @viatropos on OSX :

coffee -b -c -w -l ../
/Users/***/Sites/***/tmp/bin/nodejs/node_modules/coffee-script/lib/coffee-script/command.js:287
      if (e.code !== 'ENOENT') throw e;
                                     ^
Error: watch EMFILE
    at errnoException (fs.js:636:11)
    at FSWatcher.start (fs.js:663:11)
    at Object.watch (fs.js:691:11)

If I run this command on a smallest folder, it's work.

[braddunbar]

@craSH, @viatropos -- I have been using ulimit -n to get around this issue for the time being.

It seems odd that system settings, i.e. file descriptor limit, should cause a scaling issue. I feel as though the program should wait until a file descriptor becomes available rather than throw. However, I'm still unsure about what level this type of error should be handled at.

[craSH]

@braddunbar different craSH - you meant @Crash-- ;) cheers!

[TrevorBurnham]

I just encountered the EMFILE error for the first time myself. I added a console.log to show which things were being watched, and the result was 211 lines of the form

.git/objects/00
.git/objects/01
.git/objects/02

and so on.

I've reported this as Node issue #2479. For now, you should work around it by only using coffee -w on directories that don't have more than a couple hundred items.

[TrevorBurnham]

Ben Noordhuis has stated that there's nothing Node (or CoffeeScript) can do to work around this limit; you can modify it at the OS level using ulimit.

So, closing this issue. We should probably ignore hidden subdirectories to get around the git repo case.

[jashkenas]

Reopening to ignore hidden subdirectories ;)

[michaelficarra]

Closed by 127653b.

[bzarzuela]

I'm experiencing this with version 1.4.0. In the directory I'm trying to watch, there is a symlink to another directory. Total number of files is less than 300. If I remove the symlink, the problem goes away. Using OSX Mountain Lion.