Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure TiffReader to read ZYX chunks at a time #132

Closed
evamaxfield opened this issue Jun 24, 2020 · 1 comment
Closed

Configure TiffReader to read ZYX chunks at a time #132

evamaxfield opened this issue Jun 24, 2020 · 1 comment
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@evamaxfield
Copy link
Collaborator

Use Case

Please provide a use case to help us understand your request in context

I was watching a dask talk given by one of the core devs of dask / distributed and heard an interesting tidbit of information which was basically "try to make the tasks on your graph be more than about 200ms as if it is less than that the scheduler overhead is too large to really warrant the dask concurrency" or something similar to that. Basically, because our chunks are so small they do typically return near the 200ms mark on a good day.

By finally fixing this issue to allow for chunking of arbitrary dims for tiffs we are both solving the above issue but also drastically reducing the number of tasks placed on the scheduler / graph.

Solution

Please describe your ideal solution

Same interface as CziReader / LifReader, chunk_by_dims exposed.

Alternatives

Please describe any alternatives you've considered, even if you've dismissed them

@evamaxfield evamaxfield added the enhancement New feature or request label Jun 24, 2020
@evamaxfield evamaxfield self-assigned this Jun 24, 2020
@evamaxfield evamaxfield added this to the Release 4.0 milestone Jun 26, 2020
@evamaxfield evamaxfield removed this from the Release 4.0 milestone Nov 17, 2020
@evamaxfield evamaxfield added the help wanted Extra attention is needed label Nov 17, 2020
@evamaxfield evamaxfield removed their assignment Dec 12, 2020
@evamaxfield
Copy link
Collaborator Author

More up-to-date conversation about this in #178

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant