[INTEGRATION] with AiStore #1482
Description
🚨🚨 Feature Request
- Related to an existing AIStore example added examples#15
Is your feature request related to a problem?
Add integration to aistore as a storage provider. Boto3 can not work with AiStore.
If your feature will improve HUB
Hub will support Nvidia AiStore as backend storage.
Description of the possible solution
- Add a storage provider in https://github.com/activeloopai/Hub/tree/main/hub/core/storage similar to S3, but with gopen implementation as seen in Webdataset.
- Add a unit test
- Make sure examples pull request work perfectly.
Notes
- Boto3 could not work with aistore due to this issue https://github.com/NVIDIA/aistore/blob/master/docs/s3compat.md#boto3-compatibility