Better dask wrapper that obeys NEP18 #26
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@VolkerH
Here's a solution that I think works well now. It's actually a subclass of dask.Array, and it passes all the tests.
It's significantly faster than not using the subclass (bench below) so I'm going to leave it as the default, but now there's an
opening_array
parameter that you can set to False to get a pure dask.Array.Also, I added logic to the
_dask_block
function that will re-open/close the file if it's closed. So, it will no longer fail to useopening_array = False
... even if the underlying resource has been closed. It will just be slower... (you can speed it up again by using your own context manager to re-open the file before computing on the array).lemme know how it works for you, and if you have any suggestions/requests