Open
Description
Is your feature request related to a problem? Please describe.
We'd like to have download_chunks_concurrently
and upload_chunks_concurrently
support file-like objects. That way, we can upload the model checkpoints directly from memory instead of writing them to disk first.
Describe the solution you'd like
Support passing a file-like object, such as BytesIO
, to the filename
in the function's argument.
Describe alternatives you've considered
Due to the lack of this functionality, I need to write my data in a local file and then upload the file to object storage.
Additional context
N/A