-
Notifications
You must be signed in to change notification settings - Fork 0
Closed
Labels
area/cloudCloud provider integrationsCloud provider integrationsarea/storageStorage layer and backendsStorage layer and backendspriority/highHigh priorityHigh prioritysize/LLarge: 1-2 weeksLarge: 1-2 weekstype/featureNew feature or functionalityNew feature or functionality
Description
Summary
Implement the Storage trait for Alibaba OSS using the object_store crate with S3-compatible API.
Parent Epic
- [Epic] Distributed Roboflow with Alibaba Cloud (OSS + ACK) #9 Distributed Roboflow with Alibaba Cloud
Dependencies
- Depends on: [Phase 1.5] Create StorageFactory for backend instantiation #25 (StorageFactory)
Tasks
- Create
src/storage/oss.rs - Define
OssStoragestruct with:store: Arc<dyn ObjectStore>from object_store crateruntime: tokio::runtime::Runtimefor blocking operationsconfig: OssConfigfor settings
- Define
OssConfigstruct with:endpoint: Stringbucket: Stringaccess_key_id: Stringaccess_key_secret: Stringregion: Stringuse_internal_endpoint: bool
- Implement
OssStorage::new(config: OssConfig):- Create tokio runtime
- Configure AmazonS3Builder with OSS settings
- Set force_path_style for OSS compatibility
- Build object_store client
- Implement
Storagetrait forOssStorage:reader(): Useget()with runtime.block_on(), wrap body in adapterwriter(): CreateOssWriterthat buffers and uploads on drop/closeexists(): Usehead()operationsize(): Usehead()operationmetadata(): Usehead()and convertlist(): Uselist()with pagination handlingdelete(): Usedelete()operationcopy(): Usecopy()operation
- Create
OssWriterstruct implementingWrite:- Buffer writes to internal Vec or temp file
- Upload on
flush()ordrop() - Handle large writes with multipart (delegate to [Phase 2.2] Implement multipart upload for large files #12)
- Handle S3 eventual consistency considerations
- Add connection pooling configuration
- Feature-gate entire module under
cloud-storage - Write integration tests with MinIO or localstack
- Document OSS-specific configuration
Acceptance Criteria
-
OssStorageimplements allStoragemethods - Works with Alibaba OSS endpoints
- Works with AWS S3 (for testing with MinIO)
- Internal tokio runtime handles async operations
- Connection pooling configured
- All integration tests pass
- Feature-gated under
cloud-storage
Files to Create
src/storage/oss.rs
Configuration
| Setting | Description | Default |
|---|---|---|
endpoint |
OSS endpoint URL | Required |
bucket |
Bucket name | Required |
access_key_id |
Access key | Required |
access_key_secret |
Secret key | Required |
region |
Region code | cn-hangzhou |
use_internal_endpoint |
Use internal network | false |
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
area/cloudCloud provider integrationsCloud provider integrationsarea/storageStorage layer and backendsStorage layer and backendspriority/highHigh priorityHigh prioritysize/LLarge: 1-2 weeksLarge: 1-2 weekstype/featureNew feature or functionalityNew feature or functionality