Piping Data Improving and Performance

Example project to unsderstand how to use streams to improve the performance. This project is part of a presentation for the DevFest 2022 organized by GDG Cali

How to use it

Install

# Install dependencies
npm i

Create input file from seed

Seed -> Ids -> Input

# Create a recipe files of 1GB
npm run seed -- 1

# Create a recipe files of 0.2GB (~200MB)
npm run seed -- 0.2

Process seed file

# Process file using the buffer solution
npm run start -- buffer

# Process file using the stream solution
npm run start -- stream

Copyright

• Example data downloaded from Datakick