Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[apache#2059] feat(client-python): Support Gravitino Virtual FileSyst…
…em in Python (apache#3528) ### What changes were proposed in this pull request? Support Gravitino Virtual File System in Python so that we can read and write Fileset storage data. The first PR only supports HDFS. After research, the following popular cloud storages or companies have implemented their own FileSystem based on fsspec(https://filesystem-spec.readthedocs.io/en/latest/index.html): 1. S3(https://github.com/fsspec/s3fs) 2. Azure(https://github.com/fsspec/adlfs) 3. Gcs(https://github.com/fsspec/gcsfs) 4. OSS(https://github.com/fsspec/ossfs) 5. Databricks(https://github.com/fsspec/filesystem_spec/blob/master/fsspec/implementations/dbfs.py) 6. Snowflake(https://github.com/snowflakedb/snowflake-ml-python), So this PR will implement GVFS based on the fsspec interface. ### Why are the changes needed? Fix: apache#2059 ### How was this patch tested? Add some UTs and ITs. --------- Co-authored-by: xiaojiebao <xiaojiebao@xiaomi.com>
- Loading branch information