ipfs-http-client migration: Additional tasks

[SgtPooki]

For the full context see ipfs/kubo#9125

Broken during migration

High priority items

• fix 'should import car with roots but no blocks' test #94
• bug: ipfs refs is broken after kubo 12 #77

Other items

• IPNS tests - feat: Re-enable interface tests and get them succeeding #83 (comment)
  • Seems to work locally, a9b2b71 (#83)
• 'should get a directory' tests - feat: Re-enable interface tests and get them succeeding #83 (comment)
  • Fixed: cb2566a (#83)
• ''should pin dag-cbor' - intermittent failures on webworkers feat: Re-enable interface tests and get them succeeding #83 (comment)
  • Unskipped, seems to be working with latest updates. 65d42f1 (#83)
• 'should default to blocking pin' - intermittently failing for chrome only. see
  • https://github.com/ipfs/js-kubo-rpc-client/actions/runs/3680953673/jobs/6227147452#step:5:1049
  • https://github.com/ipfs/js-kubo-rpc-client/actions/runs/3681663334/jobs/6228595896
  • https://github.com/ipfs/js-kubo-rpc-client/actions/runs/3681791170/jobs/6228846053#step:5:972
• 'should fail to find other peer if peer does not exist'
  • Kubo added Delegated Routing over HTTP, deprecated ipfs dht and switched to ipfs routing which uses all defined routers – probably related to these changes, Antonio may help in debugging this.
• Fix and re-enable test/node/agent.js concurrency test #98
• undici fetch causing issues. see feat: Re-enable interface tests and get them succeeding #83 (comment)
• Update config.replace interface test #97

Broken prior to migration

• fix key import. #144
  • see HTTPError: file argument 'key' is required when using go-ipfs node with ipfs-http-client js-ipfs#3547
  • Kubo supports different format than JS-IPFS – see feat(cmds): add cleartext PEM/PKCS8 for key import/export kubo#8616
  • see bug: key with invalid format during import over RPC provides unhelpful error message kubo#9508
  • PEM export not yet supported in Libp2p: Support cleartext PEM key import/export libp2p/js-libp2p-crypto#244
• fix and re-enable log.tail tests. #66
• Intermittent failures after merging feat: Re-enable interface tests and get them succeeding #83,
  • 'should respect timeout option when querying the DHT' on macos:node
    • passed in https://github.com/ipfs/js-kubo-rpc-client/actions/runs/3708552206/jobs/6286726306
  • '.pubsub.subscribe"after each" hook for ".onError when connection is closed"' on chrome:webworker
    • passed in https://github.com/ipfs/js-kubo-rpc-client/actions/runs/3708552206/jobs/6286726306

Removed tests that should be re-added

• restore mode and mtime tests
  • see feat: Re-enable interface tests and get them succeeding #83 (comment). "When Support unixfs 1.5 inline metadata go-unixfs#117 and [BOUNTY] Implement UnixFSv1.5 in go-ipfs kubo#6920 land, someone will want to restore these tests, but i think its fine to remove them for now (they will still be in git history)."

Skipped tests that are remaining skipped due to a bug (need investigation)

• Fix 'limits how many bytes to write to a file (Really large file)' test #95

[BigLep]

@SgtPooki : do we have a theory why copy/pasting the code out to js-kubo-rpc-client would be causing tests to fail (given we matched dependencies)?

[SgtPooki]

@SgtPooki : do we have a theory why copy/pasting the code out to js-kubo-rpc-client would be causing tests to fail (given we matched dependencies)?

@BigLep that's a great question.

I don't have a theory other than that one of the pinned deps from the ipfs, interface-ipfs-core, or ipfs-http-client got changed. This issue and above linked CI run was an attempt to remove any of those variables, but obviously it doesn't fix all the problems.

Other issues were called out in #5 (comment) and other comments on that thread.

There are also a few js-ipfs and proc being utilized in tests that we're now trying to test using only kubo-daemon. Are some of the tests, as written, dependent upon one of the nodes not being kubo? Probably. Do I know which ones yet? Nope.

I had a draft comment written up where I was going to show debug output from some of these failing tests and lost it. Long story short: without in-depth knowledge of whats going on in the daemon, libp2p, and the intricacies of these tests, it's going to take me a long time to solve all of these errors. That is why I argued (in kubo standup) for simply blocking them temporarily until a dev more experienced in this stack could take a look. (i.e. unblocking myself)

I can work on solving these, but I will need to bring in an expert for tests where existing debug output is not sufficient, docs are out of date, or failures are generally less obvious.

Now I'm trying to get this demo project done for next week, so I'll have to get back to diving into this later.