Skip to content

feat: change download() and download_dir() to conditionally initialize backend-specific configs#1713

Merged
gaius-qi merged 1 commit intomainfrom
feature/id
Mar 11, 2026
Merged

feat: change download() and download_dir() to conditionally initialize backend-specific configs#1713
gaius-qi merged 1 commit intomainfrom
feature/id

Conversation

@gaius-qi
Copy link
Copy Markdown
Member

Description

Refactor backend scheme constants:

  • Rename HDFS_SCHEME to SCHEME in hdfs.rs
  • Rename HUGGING_FACE_SCHEME to SCHEME in hugging_face.rs
  • Rename MODEL_SCOPE_SCHEME to SCHEME in model_scope.rs

Add Scheme::is_supported() helper method to object_storage.rs to check if a given URL scheme is a supported object storage protocol, along with corresponding unit tests.

Refactor download() and download_dir() in dfget/main.rs to conditionally initialize backend-specific configs (object_storage, hdfs, hugging_face, model_scope) based on the URL scheme instead of always populating all configs unconditionally.

Related Issue

Motivation and Context

Screenshots (if appropriate)

…alize backend-specific configs

Refactor backend scheme constants:
- Rename `HDFS_SCHEME` to `SCHEME` in hdfs.rs
- Rename `HUGGING_FACE_SCHEME` to `SCHEME` in hugging_face.rs
- Rename `MODEL_SCOPE_SCHEME` to `SCHEME` in model_scope.rs

Add `Scheme::is_supported()` helper method to object_storage.rs to
check if a given URL scheme is a supported object storage protocol,
along with corresponding unit tests.

Refactor `download()` and `download_dir()` in dfget/main.rs to
conditionally initialize backend-specific configs (object_storage,
hdfs, hugging_face, model_scope) based on the URL scheme instead of
always populating all configs unconditionally.

Signed-off-by: Gaius <gaius.qi@gmail.com>
@gaius-qi gaius-qi added this to the v2.5.0 milestone Mar 11, 2026
@gaius-qi gaius-qi self-assigned this Mar 11, 2026
@gaius-qi gaius-qi added the enhancement New feature or request label Mar 11, 2026
@gaius-qi gaius-qi enabled auto-merge (squash) March 11, 2026 04:25
Copy link
Copy Markdown

@BraveY BraveY left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov
Copy link
Copy Markdown

codecov bot commented Mar 11, 2026

Codecov Report

❌ Patch coverage is 25.00000% with 63 lines in your changes missing coverage. Please review.
✅ Project coverage is 50.29%. Comparing base (0322a7d) to head (360ebce).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
dragonfly-client/src/bin/dfget/main.rs 0.00% 62 Missing ⚠️
dragonfly-client-backend/src/hugging_face.rs 75.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1713      +/-   ##
==========================================
- Coverage   50.29%   50.29%   -0.01%     
==========================================
  Files          86       86              
  Lines       21963    21977      +14     
==========================================
+ Hits        11046    11053       +7     
- Misses      10917    10924       +7     
Files with missing lines Coverage Δ
dragonfly-client-backend/src/hdfs.rs 40.90% <100.00%> (ø)
dragonfly-client-backend/src/lib.rs 96.30% <100.00%> (ø)
dragonfly-client-backend/src/model_scope.rs 42.72% <100.00%> (-0.26%) ⬇️
dragonfly-client-backend/src/object_storage.rs 88.77% <100.00%> (+0.14%) ⬆️
dragonfly-client-backend/src/hugging_face.rs 35.49% <75.00%> (-0.32%) ⬇️
dragonfly-client/src/bin/dfget/main.rs 49.33% <0.00%> (-0.26%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@gaius-qi gaius-qi merged commit f38ce30 into main Mar 11, 2026
17 of 18 checks passed
@gaius-qi gaius-qi deleted the feature/id branch March 11, 2026 05:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants