feat: prioritize content-based task ID over blob digest in task ID calc

[gaius-qi]

Description

This pull request introduces support for task ID calculation based on blob digests for OCI registry blob downloads, which enables deduplication of blobs across registries and reduces redundant downloads and storage. It also adds a new HTTP header to control this behavior and refines the logic for task ID generation. Additionally, there are improvements to code documentation and tests.

Task ID calculation and deduplication improvements:

• Added the enable_task_id_based_blob_digest field to GetRequest and related structs, allowing the task ID to be derived from the blob digest for OCI blob URLs, which deduplicates blobs across registries. [1] [2] [3] [4]
• Introduced a new header, X-Dragonfly-Enable-Task-ID-Based-Blob-Digest, to propagate the deduplication preference through the request pipeline, and added logic to read and set this header. [1] [2] [3] [4]

Documentation and code clarity:

• Improved and clarified documentation for public functions and types in header.rs, making their purpose and usage clearer. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]

Testing:

• Added unit tests to verify correct parsing of the new X-Dragonfly-Enable-Task-ID-Based-Blob-Digest header.

Refactoring:

• Adjusted the order of precedence for task ID calculation: explicit content, then blob digest (if enabled and URL matches), then URL-based. [1] [2]

Dependency and import updates:

• Added missing import for is_blob_url to dragonfly-client-util/src/request/mod.rs.

Related Issue

Motivation and Context

Screenshots (if appropriate)

[EvanCley]

LGTM

[codecov]

Codecov Report

❌ Patch coverage is 39.65517% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 44.51%. Comparing base (d056c5b) to head (efa3d7a).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
dragonfly-client-util/src/request/mod.rs	0.00%	24 Missing ⚠️
dragonfly-client/src/proxy/mod.rs	0.00%	7 Missing ⚠️
dragonfly-client/src/proxy/header.rs	85.18%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1775      +/-   ##
==========================================
- Coverage   44.53%   44.51%   -0.02%     
==========================================
  Files          91       91              
  Lines       25790    25834      +44     
==========================================
+ Hits        11485    11501      +16     
- Misses      14305    14333      +28     

Files with missing lines	Coverage Δ
dragonfly-client/src/grpc/dfdaemon_download.rs	4.79% <ø> (ø)
dragonfly-client/src/grpc/dfdaemon_upload.rs	0.00% <ø> (ø)
dragonfly-client/src/proxy/header.rs	85.92% <85.18%> (-0.09%)	⬇️
dragonfly-client/src/proxy/mod.rs	0.00% <0.00%> (ø)
dragonfly-client-util/src/request/mod.rs	43.89% <0.00%> (-1.44%)	⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[BraveY]

LGTM