Performance of node:fs

[anonrig]

node:fs is based on FSReqCallback and makes multiple C++ calls for any operation. There are lots of places where we could just open, stat and perform the operation using only 1 C++ call.

An example PR that provided performance boost was: nodejs/node#48658.

[anonrig]

Sync methods

• fs.accessSync(path[, mode]) fs: improve error performance of sync methods node#49593
• fs.appendFileSync(path, data[, options])
• fs.chmodSync(path, mode) fs: improve error perf of sync chmod+fchmod node#49859
• fs.chownSync(path, uid, gid) fs: improve error performance of chownSync node#49748
• fs.closeSync(fd) fs: improve error performance of sync methods node#49593
• fs.copyFileSync(src, dest[, mode]) fs: improve error performance of sync methods node#49593
• fs.cpSync(src, dest[, options])
• fs.existsSync(path) fs: improve error performance of sync methods node#49593
• fs.fchmodSync(fd, mode) fs: improve error perf of sync chmod+fchmod node#49859
• fs.fchownSync(fd, uid, gid) fs: throw fchownSync error from c++ node#51075
• fs.fdatasyncSync(fd) fs: improve error performance for fdatasyncSync node#49898
• fs.fstatSync(fd[, options]) fs: improve error perf of sync lstat+fstat node#49868
• fs.fsyncSync(fd) fs: improve error performance for fsyncSync node#49880
• fs.ftruncateSync(fd[, len]) fs: improve error performance for ftruncateSync node#50032
• fs.futimesSync(fd, atime, mtime) fs: improve error perf of sync *times node#49864
• fs.lchmodSync(path, mode)
• fs.lchownSync(path, uid, gid) node:fs error path performance improvements node#49962
• fs.lutimesSync(path, atime, mtime) fs: improve error perf of sync *times node#49864
• fs.linkSync(existingPath, newPath) node:fs error path performance improvements node#49962
• fs.lstatSync(path[, options]) fs: improve error perf of sync lstat+fstat node#49868
• fs.mkdirSync(path[, options]) fs: improve error performance for mkdirSync node#49847
• fs.mkdtempSync(prefix[, options]) fs: improve error performance for fs.mkdtempSync node#49750
• fs.opendirSync(path[, options]) fs: improve error performance of opendirSync node#49705
• fs.openSync(path[, flags[, mode]]) fs: improve error performance of sync methods node#49593
• fs.readdirSync(path[, options]) fs: improve error performance of readdirSync node#50131
• fs.readFileSync(path[, options]) fs: improve readFileSync with file descriptors node#49691
• fs.readlinkSync(path[, options]) node:fs error path performance improvements node#49962
• fs.readSync(fd, buffer, offset, length[, position]) fs: improve error performance for readSync node#50033
• fs.readSync(fd, buffer[, options]) fs: improve error performance for readSync node#50033
• fs.readvSync(fd, buffers[, position]) fs: improve error performance of readvSync node#50100
• fs.realpathSync(path[, options])
• fs.realpathSync.native(path[, options])
• fs.renameSync(oldPath, newPath) fs: improve error performance for fs.renameSync node#49863
• fs.rmdirSync(path[, options]) fs: improve error performance for rmdirSync node#49846
• fs.rmSync(path[, options])
• fs.statSync(path[, options]) fs: improve error performance of sync methods node#49593
• fs.statfsSync(path[, options]) fs: improve error performance of sync methods node#49593
• fs.symlinkSync(target, path[, type]) node:fs error path performance improvements node#49962
• fs.truncateSync(path[, len]) fs: improve error performance for ftruncateSync node#50032
• fs.unlinkSync(path) fs: improve error performance for unlinkSync node#49856
• fs.utimesSync(path, atime, mtime) fs: improve error perf of sync *times node#49864
• fs.writeFileSync(file, data[, options])
• fs.writeSync(fd, buffer, offset[, length[, position]]) fs: improve error performance of fs.writeSync node#50923
• fs.writevSync(fd, buffers[, position])

[tony-go]

It's definitely a good idea to tackle syncs as we don't have to handle callbacks in c++ land as with async methods.

[CanadaHonk]

Would rimraf likely benefit from being rewritten in C++ rather than in JS using FS APIs? Potential issue/idea for the future.

[CanadaHonk]

Encoding C++ fast paths

utf8

• readFileSync fs: add a fast-path for readFileSync utf-8 node#48658
• writeFileSync fs: add c++ fast path for writeFileSync string utf8 node#49884

ascii/latin1

ascii/latin1 allows using one byte V8 strings in/out which can save string copies and potentially lead to some speedups (needs investigation).

See also: nodejs/node#49888

[benjamingr]

readfile intentionally splits the file into many reads to avoid contention but that could could live in C++ and readFile would could be a single JS->CPP call (and one CPP->JS for the callback)

[CanadaHonk]

^ fwiw this only happens in current non-fast path atm (non-utf8 encoding)

[Uzlopak]

Should we maybe put in the docs, that using a number instead of a string for mode is significantly faster?

nodejs/node#49756 (comment)

[Uzlopak]

My benchmarks show that using a string for mode is about 12 Million ops/s and a number about 250 Million ops/s. My suggestion for string in the above comment makes it about 80 Million ops/s for but is still slower than simply to use a number.

[CanadaHonk]

It could also be interesting to make a Node option/something which would log/warn when missing potential faster internal/FS paths (eg readFileSync utf8 encoding, or using a string as a mode, ...)

[aduh95]

Should we maybe put in the docs, that using a number instead of a string for mode is significantly faster?

My two cents on this: we should avoid putting performance recommendation in there as it's likely to get outdated without us noticing. We could update the examples to use a constant if they are not already, so it nudges the reader to use the (currently) faster alternative.

[CanadaHonk]

Related a bit, I noticed we use strings as default flag for some common FS ops, it might be a bit faster to use numbers instead?

[Uzlopak]

There are two params. flag and mode. I just found test-fs-existssync-false.js where we use '0777' instead of 0o777. This could be changed to avoid the conversion bottleneck.

What do you refer to?

[CanadaHonk]

I meant flag not mode yes, readFileSync uses { flag: 'r' } as an example. I wonder if doing eg { flag: constants.O_RDONLY } would have a noticable change, but it might not as it only really uses a switch with the string which shouldn't have much of an impact.