IPFS protocol for querying block data

[gavofyork]

Substrate should provide an IPFS implementation over its libp2p to allow for querying certain data from chains. A (the?) offchain worker local storage should be able to be given data to store which can be indexed quickly with a Blake2 hash. The hash should be able to be used to query via IPFS (or is it bitswap protocol?) over libp2p in order to retrieve its preimage.

Rough API endpoints:

Two new OCW APIs:

• publish(data: &[u8], until: BlockNumber) -> [u8; 32]
• unpublish([u8; 32])

Preimage lookup over libp2p for anything published up until block number until and not since unpublished.

Usecase: it should be possible for any IPFS browser node to locate and retrieve unexpired data published through an OCW pubish call.

[expenses]

@dvc94ch How in-development is libp2p-bitswap? I have a branch here: https://github.com/paritytech/substrate/tree/ashley-bitswap that attempts to integrate it into substrate, is libp2p-bitswap stable enough for us to do this, or does it need more work?

[dvc94ch]

It works ok, although I do need to spend some time on it. There are optimizations in https://blog.ipfs.io/2020-02-14-improved-bitswap-for-container-distribution/ that aren't implemented, and there's there will likely be incompatibilities once you reach a high throughput as currently the messages aren't split into whatever the maximum message size is.