📄 readline-pager

⚡ Memory-efficient, paginated file reader for Node.js with async iteration, prefetching, backward reading and optional worker support. readline-pager reads large text files page-by-page without loading the entire file into memory.

• ✅ Zero dependencies
• ✅ Async iterator (for await...of) + manual next() API
• ✅ Forward & backward reading (EOF → BOF)
• ✅ Optional worker thread mode (forward only)
• ✅ Up to ~3x faster than Node.js readline
• ✅ ~97% test coverage & fully typed (TypeScript)

Important: Performance is heavily dependent on the chunkSize option; ensure you fine-tune it for your specific I/O hardware. A setting of 64 KB is typically a good starting point. Increasing it might gradually improve read speeds, usually reaching an optimal peak depending on your hardware's capabilities.

---

📦 Installation

npm install readline-pager

---

🚀 Quick start

import { createPager } from "readline-pager";
// const { createPager } = require("readline-pager");

const pager = createPager("./bigfile.txt");

for await (const page of pager) {
  console.log(page[0]); // first line of the current page
}

---

Recommended for highest throughput:

while (true) {
  const page = await pager.next();
  if (!page) break;
}

// or
let page;
while ((page = await pager.next()) !== null) {}

• while + next() is the fastest iteration method (avoids extra async-iterator overhead).
• for await of is more ergonomic and convenient.

---

⚙️ Options

createPager(filepath, {
  chunkSize?: number,     // default: 64 * 1024 (64 KiB)
  pageSize?: number,      // default: 1_000
  delimiter?: string,     // default: "\n"
  prefetch?: number,      // default: 1
  backward?: boolean,     // default: false
  useWorker?: boolean,    // default: false (forward only)
});

• chunkSize — number of bytes read per I/O operation.
• pageSize — number of lines per page.
• delimiter — line separator.
• prefetch — max number of pages buffered internally. Not required for typical use; tuning has little effect once the engine is optimized.
• backward — read file from end → start (not supported with useWorker).
• useWorker — offload parsing to a worker thread (forward only).

---

📚 API

pager.next(): Promise<string[] | null>

Returns the next page or null when finished. Empty lines are preserved.

Note: Unlike Node.js readline, which may skip empty files or leading empty lines, readline-pager always returns all lines.

• A completely empty file (0 bytes) produces [""] on the first read.
• A file with multiple empty lines returns each line as an empty string (e.g., ["", ""] for two empty lines). Node.js readline may emit fewer or no line events in these cases.

pager.close(): Promise<void>

Stops reading and releases resources immediately. Safe to call at any time.

Read-only properties

• pager.lineCount — lines emitted so far
• pager.firstLine — first emitted line (available after first read)
• pager.lastLine — last emitted line (updated per page)

---

📊 Benchmark

Run the included benchmark:

# default run
npm run benchmark

# or customize with args
node test/_benchmark.ts --lines=20000 --page-size=500 --backward

Test setup: generated text files with uuid, run on a fast NVMe machine with default options; values are averages from multiple runs. Results are machine-dependent.

The Average Throughput (MB/s) is computed for two strategies: reading files line by line and page by page.

In addition to Node, the two other popular JavaScript runtimes were also tested with readline-pager.

Line by line

Runtime / Method	1M lines (35 MB)	10M lines (353 MB)	100M lines (3,529 MB)	1,000M lines (35,286 MB)
Node — node:line	369	435	455	455
Deno — node:line	203	230	230	229
Deno — deno:line	738	901	915	809
Bun — node:line	246	279	283	280
Bun — bun:line	938	1,540	1,668	1,315

Page by page

Runtime / Method	1M lines (35 MB)	10M lines (353 MB)	100M lines (3,529 MB)	1,000M lines (35,286 MB)
Node — readline-pager	1,053	1,311	1,278	936
Deno — deno:page	852	909	908	783
Deno — readline-pager	1,131	1,268	1,271	911
Bun — bun:page	411	440	449	428
Bun — readline-pager	827	1,021	1,040	804

Runtime Environment: Node.js v25.6.1 & Bun v1.3.9 & Deno 2.6.10

---

🛠 Development & Contributing

• Minimum supported Node.js: v18.12 (lts/hydrogen).
• Development/test environment: Node v25.6 & TypeScript v5.9.

Run tests:

npm ci
npm test

Contributions are welcome — feel free to open an issue or PR.

---

📜 License

MIT — © Morteza Jamshidi