Skip to content

Enable common datasource support for Azblob, Oss, and Gcs storage #6105

@fengjiachun

Description

@fengjiachun
          @fengjiachun 

just check the code and do the tests,
about copy parquet part for now its only support s3 and fs

pub fn build_backend(url: &str, connection: &HashMap<String, String>) -> Result<ObjectStore> {
let (schema, host, path) = parse_url(url)?;
let (root, _) = find_dir_and_filename(&path);
match schema.to_uppercase().as_str() {
S3_SCHEMA => {
let host = host.context(error::EmptyHostPathSnafu {
url: url.to_string(),
})?;
Ok(build_s3_backend(&host, &root, connection)?)
}
FS_SCHEMA => Ok(build_fs_backend(&root)?),
_ => error::UnsupportedBackendProtocolSnafu {
protocol: schema,
url,
}
.fail(),
}
}

So I think just support s3 is OK and maybe support others in the future

Originally posted by @yihong0618 in #5585 (comment)

Currently, both copy from/to table/database and external table utilize build_backend to create object storage, but only S3 is supported at this time. We need to extend support to other types of object storage.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions