Skip to content

fs#opendir does not return all files in subdirectories when bufferSize is less than the directory size #48820

@mdouglass

Description

@mdouglass

Version

Node.js v20.4.0

Platform

Linux rowlf 6.3.11-200.fc38.x86_64 #1 SMP PREEMPT_DYNAMIC Sun Jul 2 13:17:31 UTC 2023 x86_64 GNU/Linux

Subsystem

fs

What steps will reproduce the bug?

If I call fs#opendir in recursive mode on a directory structure where a subdirectory has more than bufferSize files in it, than the complete results will not be returned (each subdirectory gets truncated to bufferSize entries.

I have attached a code sample that demonstrates the bug. It requires a directory structure with at least 64 files in a subdirectory of a directory named to-read:

.
├── repro-opendir.mjs
└── to-read
    └── subdir
        ├── 1
        ├── 2
        ├── ...
        └── 64

I created the above on my machine with for i in $(seq 1 64); do; touch $i; done

repro-opendir.mjs:

import assert from "assert";
import fs from "fs/promises";

async function opendir(dir, bufferSize) {
  const files = [];
  for await (const file of await fs.opendir(dir, { recursive: true, bufferSize })) {
    files.push(file);
  }
  return files;
}

async function main() {
  const filesA = await opendir("./to-read", 128);
  assert.strictEqual(filesA.length, 65);

  const filesB = await opendir("./to-read", 32);
  assert.strictEqual(filesB.length, 65);
}

await main();

How often does it reproduce? Is there a required condition?

With the directory structure as described it reproduces 100% on node.js v20.3 and v20.4. I have not tried other versions.

What is the expected behavior? Why is that the expected behavior?

Neither assertion should trigger.

What do you see instead?

With the above code sample, the second assert will fail with the following message:

AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:

33 !== 65

    at main (file:///home/matthew/spikes/repro-node-opendir/repro-opendir.mjs:17:10)
    at async file:///home/matthew/spikes/repro-node-opendir/repro-opendir.mjs:20:1

Additional information

No response

Metadata

Metadata

Labels

fsIssues and PRs related to the fs subsystem / file system.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions