Implement go-ipld-prime readable and writable storage API (v2/storage)

[rvagg]

Primary goals

• Implement the github.com/ipld/go-ipld-prime/storage APIs on top of a persistent CAR.
• Minimal use of io interfaces for each use-case, don't directly manage os.File lifecycles (i.e. closing is not a concern of this API)
• A streaming writable CARv1 implementation of WritableStorage
• Implementation of StreamingReadableStorage that pulls out io.Readers from GetStream() calls to read directly from the underlying CAR data. This is what Implement a streaming read only IPLD native storage #349 was trying to do but on top of blockstore. I've pulled that commit in for posterity although I've not really used any of it.
• No (direct) use of older IPFS packages (particularly the ones likely to get pulled into go-libipfs); the intention is for this to feel go-ipld-prime native.

A use-case I have for this (will follow-up with a link to a Lassie PR) is to have read/write block storage and a streaming writable interface. I can do this nicely with this branch by tying two separate WritableStorage outputs together to the same LinkSystem.

Secondary items

• Implemented in a v2/storage package; no references to
• Refactored most of the shared logic with blockstore into internal/store and did a bit of cleanup in the process.
• Pulled up the blockstore Options into the v2 so they can be shared, leaving var aliases for them so they shouldn't break for existing users.
• Implemented a custom ErrNotFound instead of using the go-ipld-format one (see last primary goal). This might be a bit controversial but in doing so I'm also documenting that users should prefer feature rather than type detection; they both have a IsNotFound() bool on them which should be used instead. I've debated proposing we add this to go-ipld-prime/storage, maybe that's the next iteration here but the spirit of go-ipld-prime suggests that it may not fit so well there? It doesn't have an opinion (yet) on "not found" as a distinction.

Other implementation notes

• I've gone for NewX() to denote the creation of a fresh thing, so these don't use resumption logic, they start from scratch (NewWritable() and NewReadableWritable()). And OpenX() to denote something existing / resumed (OpenReadable() and OpenReadableWritable() - the resume version of NewReadableWritable()).
• There's the awkward quick of resumption of CARv2 contents where you need to truncate the file to avoid possible corruption. I've done some feature-detection here so if you are writing a v2 and do an OpenReadableWritable() then you'll error if you can't Truncate() the IO object you've provided (i.e. you should be providing a File, but this doesn't preclude some other interface with that superpower). But you won't get an error for the same call where you request a v1.
• Marked this as "EXPERIMENTAL" in the docs for it, giving us permission to introduce breaking changes in a semver-minor.
• There's currently no key/block iteration method on these. That's not part of the needed go-ipld-prime/storage interfaces but is probably going to be helpful at some point.

[rvagg]

filecoin-project/lassie#69 for the initial use-case for this. See StreamingStore using both a NewReadableWritable() and a NewWritable() for the same content.

[willscott]

I'm happy with this refactoring direction

[hannahhoward]

Cursory glance suggests this is a very good refactor. That said, I'm not going over with a fine tooth comb so probably make sur ethere is good test coverage given the size of the change.

[rvagg]

probably make sure there is good test coverage given the size of the change.

there's better test coverage of this than the blockstore code

[rvagg]

will follow up a merge with a v2.7.0 release