Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better dask wrapper that obeys NEP18 #26

Merged
merged 3 commits into from
Nov 12, 2021
Merged

Better dask wrapper that obeys NEP18 #26

merged 3 commits into from
Nov 12, 2021

Conversation

tlambert03
Copy link
Owner

@VolkerH

Here's a solution that I think works well now. It's actually a subclass of dask.Array, and it passes all the tests.
It's significantly faster than not using the subclass (bench below) so I'm going to leave it as the default, but now there's an opening_array parameter that you can set to False to get a pure dask.Array.

Also, I added logic to the _dask_block function that will re-open/close the file if it's closed. So, it will no longer fail to use opening_array = False ... even if the underlying resource has been closed. It will just be slower... (you can speed it up again by using your own context manager to re-open the file before computing on the array).

lemme know how it works for you, and if you have any suggestions/requests

@codecov
Copy link

codecov bot commented Nov 11, 2021

Codecov Report

Merging #26 (72c062a) into main (c882be6) will increase coverage by 0.63%.
The diff coverage is 97.56%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #26      +/-   ##
==========================================
+ Coverage   86.66%   87.29%   +0.63%     
==========================================
  Files          11       11              
  Lines        1117     1149      +32     
==========================================
+ Hits          968     1003      +35     
+ Misses        149      146       -3     
Impacted Files Coverage Δ
src/nd2/nd2file.py 91.82% <96.15%> (+2.03%) ⬆️
src/nd2/opening_dask_array.py 98.21% <98.21%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c882be6...72c062a. Read the comment docs.

@tlambert03 tlambert03 mentioned this pull request Nov 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant