Using fsspec to download files

[samsja]

🚀 Feature

I was wondering if you planned on using fsspec for downloading file. Since fsspec is already compatible with many cloud provider it would make lit-data streaming compatible with even more cloud.

Motivation

Right now there is one downloader class for each provider, fsspec would allow to abstract it away, reducing the amount of cloud provider specific code and allowing supporting more out of the box

PS: love the work on lit data

[github-actions]

Hi! thanks for your contribution!, great first issue!

[tchaton]

Hey @samsja, feel free to contribute fsspec support

[samsja]

Hey @samsja, feel free to contribute fsspec support

Sure, I can give it a try

[deependujha]

Hey @samsja, there has been no update from your side regarding any progress.

So, I'd like to work on this.

If you have any issue, plz let me know.

[samsja]

Hey @samsja, there has been no update from your side regarding any progress.

So, I'd like to work on this.

If you have any issue, plz let me know.

Unfortunately, I don't have time right now to check this. Feel free to take it