Repository navigation
readline sometimes produces lines with \r in them #45992
Description
Activity
Do you set
crlfDelayoption? The recommended value for files isInfinityas shown in the doc example. FWIW if you usefilehandle.readLines(), you won't have to worry about this.Thanks for looking into this! I do have
crlfDelayset toInfinity. I'm usingcreateInterfacebecause I don't want to restrict the library to just files. The source is a simplecreateReadStream.Can you open a PR to add a failing test case to
test/known_issues? Or if you’re able to fix the issue yourself that’s even better of course, but having a test would already be a big help :)- added a commit that references this issue
on Jan 4, 2023 - added a commit that references this issue
on Jan 22, 2023 Interestingly, it "just" works if the input file has 0x3333 lines, but fails as soon as it has 0x3334 or more. I'm investigating.
The lines are each 5-byte long in the test case, and
0x3333*5 = 0xffff. If I change the test to use 4-byte long lines, it no longer reproduce. My guess is that having a carriage return on position 0xffff is problematic.0xffff = (64 * 1024) - 1, and64*1024is the default highWaterMark according to https://nodejs.org/api/fs.html#fscreatereadstreampath-options.By using
fs.createReadStream(filepath, { highWaterMark: 6 }), I'm able to reproduce the issue from thefirstsecond line.EDIT: I've simplified the test case to be minimal, and opened #46306 with a fix.
Reacted by Alec Mev- added a commit that references this issue
on Jan 22, 2023 - added a commit that references this issue
on Jan 24, 2023 - added 2 commits that reference this issue
on Feb 1, 2023
Version
19.3.0
Platform
6.1.1-arch1-1 x86_64 GNU/Linux
Subsystem
readline
What steps will reproduce the bug?
Unfortunately, the files
readlinehas trouble with are proprietary and I can't share them. I have some ideas on how to craft one, but it would take some time, so I'd like to see first if it will be needed at all.How often does it reproduce? Is there a required condition?
Consistently. Isn't present in 19.2.0 and earlier.
What is the expected behavior?
No
\rin any of the lines.What do you see instead?
Some of the files (not all) produce 1-3 lines with an
\rin it. In one file it happens 21870 lines in (109243 preceding characters), in another 75432, in another 120396. I see no pattern, but it's deterministic.Additional information
Here are a few lines from one of the files:
The lines are
\r\n-delimited. Could be caused by #45614? Not sure why only a couple of lines get affected.