Open
Description
Specification
The streams modules are still based on readable-stream 3.6.0, and also it ported from VFS but the implementation had gone through changes that haven't been fully verified. We should upgrade the readable-stream to 4.x.x https://www.npmjs.com/package/readable-stream which is cut from nodejs v18. Then refactor our streams abstraction, and also write tests to verify all the functionality.
Of particular note is the fact that our _destroy
doesn't seem to look right. There's alot of callbacks and errors that are being threaded around.
Additional context
- ci: merge staging to master #74 had to deal with the streams not properly asynchronously closing the streams and file descriptors
Tasks
- Upgrade to 4.x.x of readable-stream
- Review https://nodejs.org/api/stream.html#implementing-a-writable-stream and compare with our current implementation
- Consider reviewing the source of readable stream for the default implementation, as the main thing is the opening and closing of our EFS file descriptors