Open
Description
I did a bit of investigation into wrapping the existing kerchunk grib reader to create a GRIB backend but discovered that kerchunk
forces inlining of derived coordinates that are not stored at the message level. Virtualizarr
does not support reading inlined refs. This might be a good reason to kick off work on a Virtualizarr
specific backend without a direct kerchunk dependency (which was an eventual goal).
I personally have limited experience working with GRIB internals so it would be valuable to get input here from someone with deeper experience like @mpiannucci. A few questions
- Should we consider https://github.com/mpiannucci/gribberish for
ChunkManifest
generation (will it see continued development/maintenance)? - I'm leaning towards a model where we use
gribberish
as an optional dependency inVirtualizarr
and place the backend code in this project rather than generatingChunkManifests
viagribberish
as is currently done for kerchunk refs viascan_gribberish
but I don't have strong opinions on this. - Can we assume coordinate alignment across all messages in a GRIB file and use
open_virtual_datatree
or should our we also include anopen_virtual_groups
method for problematic datasets? @mpiannucci we'd also probably need some recommendations for from you about documentation we can include on the types of concat and grouping operations users would perform on thedict
returned byopen_virtual_groups
.