Implement a streaming read only IPLD native storage

[hannahhoward]

Goals

Start to implement CARv2 IPLD Storage native interfaces

Implementation

So this is definitely just an experiment, but I think it revealed some tangled dependencies in go-car/v2 blockstore interfaces.

Essentially, all I wanted to do was implement a go-ipld-prime ReadableStorage interface as defined here but I also wanted support for native read streaming -- https://github.com/ipld/go-ipld-prime/blob/master/storage/api.go#L91 -- cause I think this is an interesting use case.

To implement the streaming GetStream I needed lower level access to the car components, but if I implemented in a seperate package it was tough to get all the functionality of the blockstore interfaces.

I think maybe there's common wrapper code to extract? I dunno I didn't want to do a bunch of refactors.

Ultimately, this was for a particular use case I wanted to try -- I'm not convinced it can be easily written outside the go-car/v2 package, but I'm not not convinced it shouldn't be done that way.

[rvagg]

this is the bit that prevents you from doing this elsewhere isn't it?

[hannahhoward]

yes.

in fairness, the same weakness applies to most of our existing storage system cause we've never designed them around this kind of access.

[willscott]

this does seem pretty similar to readonly.go in the same package - I'm not sure I fully understand why the extra methods for streaming can't be added there?
The interface for ReadableStorage is pretty new and didn't exist at the time that this blockstore package was written, which is why it has the lower level interface of a direct blockstore - another option there is to use the storage/bsadapter shim provided in ipld-prime

[rvagg]

done in v2.7.0 in the v2/storage package