Open
Description
openedon Aug 12, 2021
Currently the storage backend API isn't very formalized and it has some problems which would be nice to fix.
- Firstly, the backend entrypoint it doesn't support
ResourceContexts
natively, instead it's just a single method which provides the user's callback with an open "dataset root". In Use ResourceContexts.jl for resource handling #12 I needed to inventResourceContexts.enter_do
to expose the data handle from within the do block butenter_do
is kind of hacky/complex/inefficient as it needs to use a separate task stack. - Secondly, the "dataset root" type which is opened by the entrypoint has an informally defined API for use with our
BlobTree
abstraction; it's kind of complex and poorly distinguished from the internals ofBlobTree
itself, which makes it hard to implement and leads to suboptimal code reuse between backends.
Likely we should have an AbstractDataStorage
type and perhaps define an open_storage
function to go with this to solve the first problem. The second problem needs some joint refactoring of the existing tree storage backends to extract the common code.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Metadata
Assignees
Labels
No labels