HTTP Gateway fails to load sharded website

[lidel]

Part of an effort to run embedded js-ipfs in Brave 🦁 ipfs/ipfs-companion#716

• Version: js-ipfs version: 0.35.0-rc.3
• Platform: Linux, Node.js v10.15.0
• Subsystem: HTTP Gateway

Type: Feature/Bug?

Severity: High?

Description:

Tried to open wikipedia mirror from js-ipfs's Gateway and got sad. Not sure if a bug or just not implemented?

Steps to reproduce the error:

1. Open http://127.0.0.1:9090/ipfs/bafybeiemxf5abjwjbikoz4mc3a3dla6ual3jsgpdr4cjr3oz3evfyavhwq/
2. Get sad

$ curl http://127.0.0.1:9090/ipfs/bafybeiemxf5abjwjbikoz4mc3a3dla6ual3jsgpdr4cjr3oz3evfyavhwq/wiki/
{"statusCode":404,"error":"Not Found","message":"no link named \"wiki\" under bafybeiemxf5abjwjbikoz4mc3a3dla6ual3jsgpdr4cjr3oz3evfyavhwq"}

[alanshaw]

@achingbrain can you help with this one?

[achingbrain]

I think this error comes from this line in js-ipfs-http-response which is called by the gateway.

It looks like it does a bunch of DAG manipulations assuming it's dealing with a straight dag-pb node. That is I'm not sure this has ever worked with sharded directories.

It should probably use the unixfs-exporter instead as that can handle more types of node.

[achingbrain]

Actually I've just put a fix together for this, I'll submit a PR. We should be able to remove js-ipfs-http-response as a dep too so hooray for ever so slightly smaller bundle sizes.

[lidel]

@achingbrain Thank you for looking into this.

I think js-ipfs-http-response continues to be responsible for generating HTML of directory listings, and it is also a dep of service-worker-gateway – is it possible to fix it upstream, or is refactor too big to be worth it?

That is I'm not sure this has ever worked with sharded directories.

Ok, I've found this TODO in libdweb branch of Companion, so it seems it never worked.