[Enhancement] Bare hash wantlist

[bbappserver]

It is entirely possible, especially as IPFS is not yet widely adopted that a peer may have a file, but does not know a corresponding IPFS url, because it just stores the full hash of the file in some file catalog(rather than a pointer to a magic DHT shard, as seems to be the case in bitswap).

To this end I think a peer should also be able to list want hashes to a connected peer.

Enquiry Setup

1. A peer enquires to connected hosts about what form(s) of catalog hash they have (md5,sha1,sha512). [optional]
2. The peer produces a bare hash wantlist of the form (hash_type_multicodec,hash)

Peering Algorithm

On another peer

1. Read wantlist of peer p
2. Ask a bare_wantlist_provider if that file is on file
3. ipfs add --nocopy <file> if available
4. Notify peer p of:

• The corresponding ipfs url, or
• File Unavailable

Bare wantlist provider

A bare wantlist provider is a program registered with the ipfs daemon. Some configuration or environment variable will be set to its path, and accepted hash types.

The wantlist provider will be invocable in the form
cmdName <hashName>://<hash>
And shall print the path of the corresponding file, then exit 0
or
If the file is not found (or any other condition for which the peer does not want to serve the file) exit 1.

[Stebalien]

I've moved this to IPFS notes as this would be a new protocol, not just an extension to bitswap. You can probably even implement it entirely outside of IPFS:

1. You can use the ipfs p2p command to listen on/connect to peers using a custom protocol.
2. On demand, you could add files to IPFS.
3. You'd then return the final CID over this custom protocol.
4. The requester would then fetch the file through IPFS normally.

---

Note: the tricky part about this is trust. You'd have to download the entire file before you'd be able to verify that you got the right file. At the moment, we chunk files into small blocks so we can verify large files piecewise.

[bbappserver]

Well yes the trust is an interesting point. But you could argue the same about how you got some other IFPFs url in the wild. In both instances you are trusting the CID.

The idea of it being in bitswap is you could piggyback on the ledger so if someone claimed to have a file and produced something different, you could dock them points. Though I suppose the custom protocol could run a separate ledger for "got correct file" as opposed to "got correct chunk".

[Stebalien]

But you could argue the same about how you got some other IFPFs url in the wild. In both instances you are trusting the CID.

In bitswap as-is, a human has made a decision to visit some website/resource. In this case, bitswap would be trusting a random peer. Anyone could join the network and make false claims and everyone would believe them (without a reputation system).

The idea of it being in bitswap is you could piggyback on the ledger so if someone claimed to have a file and produced something different, you could dock them points. Though I suppose the custom protocol could run a separate ledger for "got correct file" as opposed to "got correct chunk".

Ah, got it. Really, I'd love to eventually extract the ledger out into a peer reputation module where any service could ask about and report on reputation/behavior of peers.