fix(proxy): preserve SigV4-signed ranges

[YQ-Wang]

Description

This fixes multipart S3 downloads through dfdaemon's HTTP proxy when AWS SigV4 or SigV4a covers the Range header.

The change is limited to signature-bound ranges:

• Detect Range in SigV4/SigV4a signed headers, including presigned URLs.
• Preserve the caller's exact signed range across stat and source-piece requests.
• Reject signature-bound multi-range requests because dfdaemon cannot produce a multipart/byteranges response without changing the signed request.
• Derive stable, compact range task IDs from the normal task ID, requested range, and representation-selecting headers while excluding rotating credentials.
• Include Accept, Accept-Language, and Accept-Encoding in compact-range cache identity to avoid cross-representation cache reuse.
• Store each signed range at local offset zero and translate the proxy response back to source coordinates.
• Allow small signed ranges to use the scheduler, peers, and local cache.
• Validate 206 Partial Content, Content-Range, Content-Length, ETag, and S3 version ID when present before caching source bytes.
• Keep download and upload paths consistent for compact range tasks.
• Send signed If-Range requests directly to origin because a validator mismatch can legitimately return a full 200 response that a compact range task cannot represent.
• Additionally limit inline gRPC piece content to 64 MiB to avoid a very large protobuf allocation, while streaming proxy requests remain supported up to the overall 1 GiB signature-bound range limit.
• Preserve real origin error and conditional statuses, including 304, while reporting locally generated validation failures on nominal success responses as 502 Bad Gateway and correctly framing empty responses.

Unrelated S3-aware routing, TLS policy, proxy authentication, redirect, logging, and general cache-revalidation changes are intentionally excluded.

Related Issue

Fixes #1821

Motivation and Context

Multipart S3 clients such as s5cmd sign each Range request. dfdaemon normally rewrites ranges into Dragonfly piece ranges, which changes the canonical request and causes S3 to reject it with 403 SignatureDoesNotMatch.

For requests that do not explicitly sign Range, the existing download path remains unchanged.

Tests

Validated on commit 1ef00b966b55a0cf4696edc4873dffb3e6d49de2 against main at 6a9b8c184961ff1a9928d25356c27513b188d829:

• cargo fmt --all -- --check
• cargo check --all --all-targets
• cargo clippy --all --all-targets -- -D warnings
• Focused dragonfly-client library tests (66 tests)
• cargo test --all-features --workspace (461 tests, 0 failures; 4 doc tests ignored)

The exact PR head was also exercised with a Linux binary built from that commit, two dfdaemons, S3-compatible MinIO, a recording relay, and s5cmd v2.3.0 using a 24 MiB object split into five signed ranges. Direct, cold-client, seed-cache, and warm-client outputs were byte-identical. The cold client received all bytes from the seed; later seed and client downloads were local-cache hits with no ranged origin requests. The five task IDs remained stable across fresh SigV4 dates and signatures, and every ranged origin GET preserved its exact range with host;range signed. No signature or signed-response validation errors were observed.

Validation Boundaries

• The integration environment used MinIO, not an AWS S3 bucket.
• Each node that first handles a signed-range task replays the unchanged signed GET to validate metadata. In the two-daemon cold test, each range caused two validation GETs and one data GET; cached repeats caused no ranged GETs, although s5cmd still sent its normal object HEAD.
• A signed range remains one Dragonfly piece because subdividing it would require rewriting the signed header.
• Streaming proxy requests are limited to a 1 GiB signed range; inline gRPC piece content is limited to 64 MiB.
• Signed If-Range requests intentionally bypass P2P to preserve HTTP validator semantics.
• Cache identity covers the common representation selectors above, but this PR does not implement arbitrary Vary processing.
• Cache freshness and authorization retain Dragonfly's existing task lifetime and deployment trust model; this PR does not add local SigV4 verification or a new cache revalidation protocol.

AI Assistance

AI assistance was used to review and refine the scope and implementation. The final patch was checked against the linked issue and validated with the commands listed above.

Screenshots (if appropriate)

N/A

[codecov]

Codecov Report

❌ Patch coverage is 95.42218% with 45 lines in your changes missing coverage. Please review.
✅ Project coverage is 53.01%. Comparing base (6a9b8c1) to head (1ef00b9).

Files with missing lines	Patch %	Lines
dragonfly-client/src/grpc/dfdaemon_download.rs	0.00%	10 Missing ⚠️
dragonfly-client/src/grpc/dfdaemon_upload.rs	0.00%	10 Missing ⚠️
dragonfly-client/src/proxy/mod.rs	95.98%	9 Missing ⚠️
dragonfly-client/src/resource/task.rs	92.07%	8 Missing ⚠️
dragonfly-client-util/src/http/mod.rs	98.79%	4 Missing ⚠️
dragonfly-client/src/resource/piece.rs	97.64%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1820      +/-   ##
==========================================
+ Coverage   51.23%   53.01%   +1.77%     
==========================================
  Files         100      100              
  Lines       26530    27498     +968     
==========================================
+ Hits        13593    14578     +985     
+ Misses      12937    12920      -17     

Files with missing lines	Coverage Δ
dragonfly-client-backend/src/http.rs	96.13% <100.00%> (+0.54%)	⬆️
dragonfly-client-util/src/id_generator/mod.rs	98.80% <100.00%> (+0.06%)	⬆️
dragonfly-client/src/proxy/task.rs	0.00% <ø> (ø)
dragonfly-client-util/src/http/mod.rs	98.82% <98.79%> (-0.12%)	⬇️
dragonfly-client/src/resource/piece.rs	52.86% <97.64%> (+16.02%)	⬆️
dragonfly-client/src/resource/task.rs	13.54% <92.07%> (+10.40%)	⬆️
dragonfly-client/src/proxy/mod.rs	35.25% <95.98%> (+35.25%)	⬆️
dragonfly-client/src/grpc/dfdaemon_download.rs	4.92% <0.00%> (-0.06%)	⬇️
dragonfly-client/src/grpc/dfdaemon_upload.rs	0.00% <0.00%> (ø)

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.