EMFILE error when using fs.watch on many items

[TrevorBurnham]

Test case:

var fs = require('fs');

for (var i = 1; i <= 500; i++) {
  fs.writeFileSync("" + i + ".tmp", '');
}

for (i = 1; i <= 500; i++) {
  try {
    fs.watch("" + i + ".tmp", function() {});
  } catch (e) {
    console.log(i);
    process.exit(1);
  }
}

With Node 0.6.2 under OS X, I consistently get output of 250, meaning that 249 files were successfully watched, but an EMFILE error occurred when trying to watch the 250th. The pertinent part of the stack trace is

Error: watch EMFILE
    at errnoException (fs.js:605:11)
    at FSWatcher.start (fs.js:632:11)
    at Object.watch (fs.js:660:11)

Several folks have experienced the error when using CoffeeScript's watch mode, particularly in a directory that's a git repository (see CoffeeScript issue #1537).

Adding a 100ms interval between each file, I get the same result. So it seems that there's a hard limit on how many files one can watch simultaneously from a Node process on some systems. Is there some way that we can increase or avoid this limit? Pinging @bnoordhuis.

[bnoordhuis]

So it seems that there's a hard limit on how many files one can watch simultaneously from a Node process on some systems. Is there some way that we can increase or avoid this limit?

Yes, ulimit -n 16384. It's not a Node limitation, it's a per-process limit that's enforced by the operating system (presumably OS X in your case, that's the only OS I know of that defaults to 256).

[TrevorBurnham]

Ah, thanks for the clarification.

[wombleton]

I'm running node 0.6.9 on linux mint 12. ulimit -n returns 65536 so as far as I can see it's not a file handle limit. The above test fails at 116.

Replacing fs.watch with fs.watchFile works fine.

[bnoordhuis]

@wombleton: What does the strace log look like?

[wombleton]

Error: watch EMFILE
    at errnoException (fs.js:636:11)
    at FSWatcher.start (fs.js:663:11)
    at Object.watch (fs.js:691:11)

[bnoordhuis]

Heh, that's not quite what I mean. Can you gist the output of strace node script.js? If you pass -o trace.log to strace, it'll write to a file instead of the console.

[wombleton]

Ah! Now I know about strace, I had it pegged as a slightly strange contraction. Apologies.

https://gist.github.com/1707965

[bnoordhuis]

You're running into a linux-specific system limit, namely the maximum number of inotify instances.

$ sysctl -a | grep fs.inotify
fs.inotify.max_user_instances = 128
fs.inotify.max_user_watches = 8192
fs.inotify.max_queued_events = 16384

It's something that we can work around. I've opened a libuv issue for it: joyent/libuv#300.

[bnoordhuis]

Fixed in joyent/libuv@d3efefb.

[kdvolder]

I'm using fs.watch on node 0.8.2 to watch a directory tree. I'm only watching the directories/subdirectories not each individual file.

This is working beatifully on Linux (Ubuntu 10.04). But we are hitting EMFILE error on Mac OS. It looks like this error condition is documented here:

https://developer.apple.com/library/mac/#documentation/Darwin/Reference/Manpages/man2/kqueue.2.html

as happening when

   [EMFILE]           The per-process descriptor table is full.

Our application does need to work on Mac OS as well as Unix and Windows. So I wonder... is there a way to increase this 'per-process descriptor table' size to something reasonable for our node process?

Any suggestions to work around this Mac OS limit welcome.

[kdvolder]

PS: I did try a couple of google searches to find the answer to my question, but the most useful result that turned up so far has been this issue :-)

[bnoordhuis]

@kdvolder Try e.g. ulimit -n 65536. With OS X you may have to log in again after that.

[kdvolder]

Further note: ulimit command as suggested above doesn't seem to exist on Mac OS. We tried using the equivalent Mac OS command but it didn't seem to work (my guess is because the limit we are hitting is not that of 'open files' but the 'The per-process descriptor table is full.' whatever that is, seems like it is maybe a different limit.