replace minipass pipeline with simple pipe

[everett1992]

I can reliably cause tar integrity errors while downloading packages
from npm thru an http proxy that uses chunked encoding. The error seems
to be in make-fetch-happen, minipass-fetch or one of the minipass
libraries. I don't understand the root cause but I'm unable to reproduce
the issue after applying this patch.

My guess is that something in the stack expects res.body to be a
Minipass instance but not MinipassPipeline, or creating the pipeline
starts body flowing before all listeners are connected, missing some
chunks of data.

fixes npm/cli#3884

To reproduce

1. Start the chunked-encoding proxy
2. In a new terminal create a npm project
   mkdir chunked-encoding-error && cd chunked-encoding-error && echo '{}' > package.json
3. configure npm to use the registry
   npm config --location project set registry http://localhost:4000
4. remove npm cache.
5. install any package from the registry (this should fail).
   • I recommend a package without dependencies, like lodash or typescript, debugging is easier.
   • The error isn't 100% consistent, you may want to run this step in
     a loop

     while rm ~/.npm/_cacache node_modules package-lock.json -rf && npm install --no-save lodash; do; done
     

[everett1992]

I started carving chunks out of npm to create a smaller reproduction and I've reached make-fetch-happen and tar. Tomorrow I'll keep carving to try to eliminate tar.

https://github.com/everett1992/make-fetch-happen-tar-extract-error/blob/main/error.test.js

This test shows a lot of weird behavior.

• why is it only reproduced with chunked-encoding?
• why is it only reproduced when integrity is passed to fetch?
• why is size correct but integrity is wrong?
• why does piping ssri to tar.x yield incorrect integrity? How does the destination affect the source?

Switching to my branch of make-fetch-happens fixes the bug and all tests pass.

error.test.js
  res.body data events updating hash
    ✓ yields the right digest

  res.body pipe to createHash
    ✓ yields the right digest

  extract with integrity
    1) zlib: incorrect data check

  extract affects source stream?
    2) source hash is expected
    ✓ source size is expected

  extract with integrity and content-length
    ✓ succeeds

  extract without integrity
    ✓ succeeds


  5 passing (6s)
  2 failing

  1) error.test.js extract with integrity zlib: incorrect data check:
     Error: zlib: incorrect data check


  2) error.test.js extract affects source stream? source hash is expected:

      Error: source hash is expected
      + expected - actual

      -24dbddf17111f46417d2fdaa260b1a37f9b3142340e4145efe3f0937d77eb56c862d2a1d2901ca16271dc0d6335b0237c2346768a3ec1a3d579018f1fc5f7a0d
      +0fafc16d5a24c317008d3d76ac460b09fa8073918af84a979769716cb4560855f36627a02654845115bf931f99e5b883f65a6429b9a4b0367b64ee0ae16311f7

[nlf]

if i'm understanding correctly, your pull request to minipass linked above fixes this issue, is that right?

[everett1992]

Yes, isaacs/minipass#28 closes this and npm/cli#3884

[nlf]

Yes, isaacs/minipass#28 closes this and npm/cli#3884

you're awesome, thanks for taking the time to chase this one down all the way to the end. i know what these bug hunts are like so i really appreciate it!

[everett1992]

Thanks, I'm glad I followed it to the root cause instead of trying to merge this hot fix.