Skip to content

Commit aeb4de5

Browse files
jasnelltargos
authored andcommitted
fs: port SonicBoom module to fs module as Utf8Stream
As a first step to porting portions of the pino structured logger into the runtime, this commit ports the SonicBoom module to the fs module as Utf8Stream. This is a faithful port of the SonicBoom module with some modern updates, such as converting to a Class and using Symbol.dispose. The bulk of the implementation is unchanged from the original. PR-URL: #58897 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
1 parent 33728cb commit aeb4de5

18 files changed

+2888
-0
lines changed

doc/api/fs.md

Lines changed: 180 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7701,6 +7701,186 @@ added:
77017701
77027702
Type of file system.
77037703
7704+
### Class: `fs.Utf8Stream`
7705+
7706+
<!-- YAML
7707+
added: REPLACEME
7708+
-->
7709+
7710+
> Stability: 1 - Experimental
7711+
7712+
An optimized UTF-8 stream writer that allows for flushing all the internal
7713+
buffering on demand. It handles `EAGAIN` errors correctly, allowing for
7714+
customization, for example, by dropping content if the disk is busy.
7715+
7716+
#### Event: `'close'`
7717+
7718+
The `'close'` event is emitted when the stream is fully closed.
7719+
7720+
#### Event: `'drain'`
7721+
7722+
The `'drain'` event is emitted when the internal buffer has drained sufficiently
7723+
to allow continued writing.
7724+
7725+
#### Event: `'drop'`
7726+
7727+
The `'drop'` event is emitted when to maximal length is reached and that data
7728+
will not be written. The data that was dropped is passed as the first argument
7729+
to the event handle.
7730+
7731+
#### Event: `'error'`
7732+
7733+
The `'error'` event is emitted when an error occurs.
7734+
7735+
#### Event: `'finish'`
7736+
7737+
The `'finish'` event is emitted when the stream has been ended and all data has
7738+
been flushed to the underlying file.
7739+
7740+
#### Event: `'ready'`
7741+
7742+
The `'ready'` event is emitted when the stream is ready to accept writes.
7743+
7744+
#### Event: `'write'`
7745+
7746+
The `'write'` event is emitted when a write operation has completed. The number
7747+
of bytes written is passed as the first argument to the event handler.
7748+
7749+
#### `new fs.Utf8Stream([options])`
7750+
7751+
* `options` {Object}
7752+
* `append`: {boolean} Appends writes to dest file instead of truncating it.
7753+
**Default**: `true`.
7754+
* `contentMode`: {string} Which type of data you can send to the write
7755+
function, supported values are `'utf8'` or `'buffer'`. **Default**:
7756+
`'utf8'`.
7757+
* `dest`: {string} A path to a file to be written to (mode controlled by the
7758+
append option).
7759+
* `fd`: {number} A file descriptor, something that is returned by `fs.open()`
7760+
or `fs.openSync()`.
7761+
* `fs`: {Object} An object that has the same API as the `fs` module, useful
7762+
for mocking, testing, or customizing the behavior of the stream.
7763+
* `fsync`: {boolean} Perform a `fs.fsyncSync()` every time a write is
7764+
completed.
7765+
* `maxLength`: {number} The maximum length of the internal buffer. If a write
7766+
operation would cause the buffer to exceed `maxLength`, the data written is
7767+
dropped and a drop event is emitted with the dropped data
7768+
* `maxWrite`: {number} The maximum number of bytes that can be written;
7769+
**Default**: `16384`
7770+
* `minLength`: {number} The minimum length of the internal buffer that is
7771+
required to be full before flushing.
7772+
* `mkdir`: {boolean} Ensure directory for `dest` file exists when true.
7773+
**Default**: `false`.
7774+
* `mode`: {number|string} Specify the creating file mode (see `fs.open()`).
7775+
* `periodicFlush`: {number} Calls flush every `periodicFlush` milliseconds.
7776+
* `retryEAGAIN` {Function} A function that will be called when `write()`,
7777+
`writeSync()`, or `flushSync()` encounters an `EAGAIN` or `EBUSY` error.
7778+
If the return value is `true` the operation will be retried, otherwise it
7779+
will bubble the error. The `err` is the error that caused this function to
7780+
be called, `writeBufferLen` is the length of the buffer that was written,
7781+
and `remainingBufferLen` is the length of the remaining buffer that the
7782+
stream did not try to write.
7783+
* `err` {any} An error or `null`.
7784+
* `writeBufferLen` {number}
7785+
* `remainingBufferLen`: {number}
7786+
* `sync`: {boolean} Perform writes synchronously.
7787+
7788+
#### `utf8Stream.append`
7789+
7790+
* {boolean} Whether the stream is appending to the file or truncating it.
7791+
7792+
#### `utf8Stream.contentMode`
7793+
7794+
* {string} The type of data that can be written to the stream. Supported
7795+
values are `'utf8'` or `'buffer'`. **Default**: `'utf8'`.
7796+
7797+
#### `utf8Stream.destroy()`
7798+
7799+
Close the stream immediately, without flushing the internal buffer.
7800+
7801+
#### `utf8Stream.end()`
7802+
7803+
Close the stream gracefully, flushing the internal buffer before closing.
7804+
7805+
#### `utf8Stream.fd`
7806+
7807+
* {number} The file descriptor that is being written to.
7808+
7809+
#### `utf8Stream.file`
7810+
7811+
* {string} The file that is being written to.
7812+
7813+
#### `utf8Stream.flush(callback)`
7814+
7815+
* `callback` {Function}
7816+
* `err` {Error|null} An error if the flush failed, otherwise `null`.
7817+
7818+
Writes the current buffer to the file if a write was not in progress. Do
7819+
nothing if `minLength` is zero or if it is already writing.
7820+
7821+
#### `utf8Stream.flushSync()`
7822+
7823+
Flushes the buffered data synchronously. This is a costly operation.
7824+
7825+
#### `utf8Stream.fsync`
7826+
7827+
* {boolean} Whether the stream is performing a `fs.fsyncSync()` after every
7828+
write operation.
7829+
7830+
#### `utf8Stream.maxLength`
7831+
7832+
* {number} The maximum length of the internal buffer. If a write
7833+
operation would cause the buffer to exceed `maxLength`, the data written is
7834+
dropped and a drop event is emitted with the dropped data.
7835+
7836+
#### `utf8Stream.minLength`
7837+
7838+
* {number} The minimum length of the internal buffer that is required to be
7839+
full before flushing.
7840+
7841+
#### `utf8Stream.mkdir`
7842+
7843+
* {boolean} Whether the stream should ensure that the directory for the
7844+
`dest` file exists. If `true`, it will create the directory if it does not
7845+
exist. **Default**: `false`.
7846+
7847+
#### `utf8Stream.mode`
7848+
7849+
* {number|string} The mode of the file that is being written to.
7850+
7851+
#### `utf8Stream.periodicFlush`
7852+
7853+
* {number} The number of milliseconds between flushes. If set to `0`, no
7854+
periodic flushes will be performed.
7855+
7856+
#### `utf8Stream.reopen(file)`
7857+
7858+
* `file`: {string|Buffer|URL} A path to a file to be written to (mode
7859+
controlled by the append option).
7860+
7861+
Reopen the file in place, useful for log rotation.
7862+
7863+
#### `utf8Stream.sync`
7864+
7865+
* {boolean} Whether the stream is writing synchronously or asynchronously.
7866+
7867+
#### `utf8Stream.write(data)`
7868+
7869+
* `data` {string|Buffer} The data to write.
7870+
* Returns {boolean}
7871+
7872+
When the `options.contentMode` is set to `'utf8'` when the stream is created,
7873+
the `data` argument must be a string. If the `contentMode` is set to `'buffer'`,
7874+
the `data` argument must be a {Buffer}.
7875+
7876+
#### `utf8Stream.writing`
7877+
7878+
* {boolean} Whether the stream is currently writing data to the file.
7879+
7880+
#### `utf8Stream[Symbol.dispose]()`
7881+
7882+
Calls `utf8Stream.destroy()`.
7883+
77047884
### Class: `fs.WriteStream`
77057885
77067886
<!-- YAML

lib/fs.js

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -170,6 +170,11 @@ let ReadFileContext;
170170
// monkeypatching.
171171
let FileReadStream;
172172
let FileWriteStream;
173+
let Utf8Stream;
174+
175+
function lazyLoadUtf8Stream() {
176+
Utf8Stream ??= require('internal/streams/fast-utf8-stream');
177+
}
173178

174179
// Ensure that callbacks run in the global context. Only use this function
175180
// for callbacks that are passed to the binding layer, callbacks that are
@@ -3353,6 +3358,11 @@ module.exports = fs = {
33533358
FileWriteStream = val;
33543359
},
33553360

3361+
get Utf8Stream() {
3362+
lazyLoadUtf8Stream();
3363+
return Utf8Stream;
3364+
},
3365+
33563366
// For tests
33573367
_toUnixTimestamp: toUnixTimestamp,
33583368
};

0 commit comments

Comments
 (0)