sys: Use readdir withFileTypes option to skip lots of stat syscalls #35286
Conversation
|
|
This makes walking large directory trees much more efficient on Node 10.10 or later. See: https://lwn.net/Articles/606995/ https://www.python.org/dev/peps/pep-0471/ nodejs/node#22020 https://nodejs.org/en/blog/release/v10.10.0/ Signed-off-by: Anders Kaseorg <andersk@mit.edu>
andersk
force-pushed
the
readdir-withFileTypes
branch
from
96a40c3 to
bee7ee0
Compare
| function getDirectories(path: string): string[] { | ||
| perfLogger.logEvent("ReadDir: " + path); | ||
| return filter<string>(_fs.readdirSync(path), dir => fileSystemEntryExists(combinePaths(path, dir), FileSystemEntryKind.Directory)); | ||
| return getAccessibleFileSystemEntries(path).directories.slice(); |
There was a problem hiding this comment.
You don't need to slice since getAccessibleFileSystemEntries returns new array for directories..
There was a problem hiding this comment.
Not always: return emptyFileSystemEntries does not. Of course, that’s easy to change. I’ve added a commit doing so.
Edit: I’ve now removed that commit again because we didn’t want to change the type of getAccessibleFileSystemEntries (at least not in this PR); see this discussion.
andersk
force-pushed
the
readdir-withFileTypes
branch
2 times, most recently
from
2b983fa to
01aec72
Compare
My reasoning was that the |
|
@andersk I don't feel strongly either way. I agree that it's a small overhead, I just thought it seemed like a shame to pay it for all future versions (whereas an explicit version check is more likely to be cleaned up if we decide to deprecate older nodes). Either way, this is a huge improvement over the current implementation. Thanks again! |
|
The CI failure is unrelated. We're working on it. |
|
Closing and re-opening to re-trigger CI. |
andersk
force-pushed
the
readdir-withFileTypes
branch
from
01aec72 to
bee7ee0
Compare
|
Thanks, @andersk! My local benchmarking suggests this will make a big difference. And sorry about the delay in getting eyes on your PR. |
…icrosoft#35286) This makes walking large directory trees much more efficient on Node 10.10 or later. See: https://lwn.net/Articles/606995/ https://www.python.org/dev/peps/pep-0471/ nodejs/node#22020 https://nodejs.org/en/blog/release/v10.10.0/ Signed-off-by: Anders Kaseorg <andersk@mit.edu>
4 participants
This makes walking large directory trees much more efficient on Node 10.10 or later.
See:
https://lwn.net/Articles/606995/
https://www.python.org/dev/peps/pep-0471/
nodejs/node#22020
https://nodejs.org/en/blog/release/v10.10.0/