Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Downloads should use files on disk instead of bytes in RAM #37

Closed
hoh opened this issue Jul 10, 2023 · 1 comment
Closed

Downloads should use files on disk instead of bytes in RAM #37

hoh opened this issue Jul 10, 2023 · 1 comment

Comments

@hoh
Copy link
Member

hoh commented Jul 10, 2023

Downloading files currently stores everything in RAM, doing so in a new BytesIO object.

We should change the definition of the function or create a new one that saves the data on disk.

My proposal is to ask the user to pass a file-like object the function can write to.

This file-like object can be a buffer in memory, a file on disk or anything compatible with the interface.

For convenience, we could make that argument optional and return a NamedTemporaryFile if it is None.

Originally posted by @hoh in #36 (comment)

@odesenfans
Copy link
Contributor

What I told @1yam was to implement a generic function that takes a buffer, as you suggest. He already implemented it (see download_file_to_buffer().

On the other hand, I don't see the point of returning a temporary file if no buffer is specified. I think we should provide a download_to_file(file_hash, path) convenience function that uses download_file_to_buffer(). I say we leave the choice of where the file is located entirely to the user.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants