Skip to content

Make zarr download begin as soon as initial page of zarr files is obtained #1447

@yarikoptic

Description

@yarikoptic

This is continuation to work done in

where I thought that interleave() would be a generator which starts yielding without delaying for a full list of zarr files to be retrieved first. @jwodder mentioned that my assumption was wrong and that it would need additional refactoring to make it behave as desired.

While addressing this it might be worth checking if interleave could natively provide support for desired use case where it takes care about running that thread with "producer" iterator. It is somewhat similar to how I have done for ProducerConsumer in datalad where Producer is ran into a separate thread as well IIRC.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions