Skip to content

[Phase 2.1] Implement OSS/S3 backend using object_store #13

@zhexuany

Description

@zhexuany

Summary

Implement the Storage trait for Alibaba OSS using the object_store crate with S3-compatible API.

Parent Epic

Dependencies

Tasks

  1. Create src/storage/oss.rs
  2. Define OssStorage struct with:
    • store: Arc<dyn ObjectStore> from object_store crate
    • runtime: tokio::runtime::Runtime for blocking operations
    • config: OssConfig for settings
  3. Define OssConfig struct with:
    • endpoint: String
    • bucket: String
    • access_key_id: String
    • access_key_secret: String
    • region: String
    • use_internal_endpoint: bool
  4. Implement OssStorage::new(config: OssConfig):
    • Create tokio runtime
    • Configure AmazonS3Builder with OSS settings
    • Set force_path_style for OSS compatibility
    • Build object_store client
  5. Implement Storage trait for OssStorage:
    • reader(): Use get() with runtime.block_on(), wrap body in adapter
    • writer(): Create OssWriter that buffers and uploads on drop/close
    • exists(): Use head() operation
    • size(): Use head() operation
    • metadata(): Use head() and convert
    • list(): Use list() with pagination handling
    • delete(): Use delete() operation
    • copy(): Use copy() operation
  6. Create OssWriter struct implementing Write:
  7. Handle S3 eventual consistency considerations
  8. Add connection pooling configuration
  9. Feature-gate entire module under cloud-storage
  10. Write integration tests with MinIO or localstack
  11. Document OSS-specific configuration

Acceptance Criteria

  • OssStorage implements all Storage methods
  • Works with Alibaba OSS endpoints
  • Works with AWS S3 (for testing with MinIO)
  • Internal tokio runtime handles async operations
  • Connection pooling configured
  • All integration tests pass
  • Feature-gated under cloud-storage

Files to Create

  • src/storage/oss.rs

Configuration

Setting Description Default
endpoint OSS endpoint URL Required
bucket Bucket name Required
access_key_id Access key Required
access_key_secret Secret key Required
region Region code cn-hangzhou
use_internal_endpoint Use internal network false

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/cloudCloud provider integrationsarea/storageStorage layer and backendspriority/highHigh prioritysize/LLarge: 1-2 weekstype/featureNew feature or functionality

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions