You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Please provide a use case to help us understand your request in context
I was watching a dask talk given by one of the core devs of dask / distributed and heard an interesting tidbit of information which was basically "try to make the tasks on your graph be more than about 200ms as if it is less than that the scheduler overhead is too large to really warrant the dask concurrency" or something similar to that. Basically, because our chunks are so small they do typically return near the 200ms mark on a good day.
By finally fixing this issue to allow for chunking of arbitrary dims for tiffs we are both solving the above issue but also drastically reducing the number of tasks placed on the scheduler / graph.
Solution
Please describe your ideal solution
Same interface as CziReader / LifReader, chunk_by_dims exposed.
Alternatives
Please describe any alternatives you've considered, even if you've dismissed them
The text was updated successfully, but these errors were encountered:
Use Case
Please provide a use case to help us understand your request in context
I was watching a dask talk given by one of the core devs of dask / distributed and heard an interesting tidbit of information which was basically "try to make the tasks on your graph be more than about 200ms as if it is less than that the scheduler overhead is too large to really warrant the dask concurrency" or something similar to that. Basically, because our chunks are so small they do typically return near the 200ms mark on a good day.
By finally fixing this issue to allow for chunking of arbitrary dims for tiffs we are both solving the above issue but also drastically reducing the number of tasks placed on the scheduler / graph.
Solution
Please describe your ideal solution
Same interface as CziReader / LifReader,
chunk_by_dims
exposed.Alternatives
Please describe any alternatives you've considered, even if you've dismissed them
The text was updated successfully, but these errors were encountered: