Repository navigation
Store: rewrite the ONTAP store as an adapter over S3Store - #2866
Open
LakshmirajSunilSawant wants to merge 1 commit into
Open
LakshmirajSunilSawant wants to merge 1 commit into
LakshmirajSunilSawant wants to merge 1 commit into
Conversation
Replace the drifted copy of s3_store.rs with a thin adapter, like the R2 and OCI stores. It keeps ONTAP's client setup (path-style, vserver as region, timeouts, custom CA) and drops the unpadded checksum header from TraceMachina#2767. Fixes TraceMachina#2779
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Member
So that's going to be a blocker here. We do need to test any changes for this against a real ONTAP store (or a NetApp-provided simulator), and figuring out access to that has been one of our maintenance problems here. |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Fixes #2779.
ontap_s3_store.rswas a 790-line fork ofs3_store.rsthat had drifted, and its hand-built checksum header was the bug in #2767. With #2768 merged, it is now a thin adapter likeR2StoreandOciStore: it builds an SDK client for ONTAP and hands it toS3Store.The client keeps what is ONTAP-specific: path-style addressing,
vserver_nameas the signing region, 30s/2min timeouts, the optionalroot_certificatesCA bundle and the default credential chain. ONTAP now getsS3Store'soptimized_for, health checks and future fixes. The existence cache now builds its listing client through the adapter, so it honoursroot_certificates.Checksums are set to
WhenRequired, as inOciStore, so uploads stay plain bodies (noaws-chunked, no checksum header). #2767 suggested "checksums on, chunked off", but the SDK only puts a checksum in a header for in-memory bodies.S3Storestreams uploads, so that would need buffering, which is left for a follow-up if ONTAP needs it.No config fields change.
how-to/stores/s3-and-compatible.mdxis updated.How was this verified?
ontap_s3_store_test.rsis rewritten in the style ofoci_store_test.rs, checking the requests on the wire with aStaticReplayClient: path-style URLs, SigV4 scope using the vserver, NotFound mapping, ranged GET, and that single and multipart uploads carry noaws-chunkedencoding or checksum headers (the regression guard for #2767). The old tests for retries, expiry and zero digests covered generic behaviour thats3_store_test.rsalready tests.cargo test -p nativelink-store, clippy and rustfmt pass locally, on Linux and on top of currentmain.Not verified against a real ONTAP endpoint, which I don't have access to. Bazel and pre-commit are left to CI.
Risk
Medium for existing ONTAP users, since it can't be checked against ONTAP before merge. Differences from the old store:
max_retry_buffer_per_requestdefaults to 5MB instead of 20MB, matchingS3Store. Single-part uploads are always under 5MB, so no practical effect.insecure_allow_httpanddisable_http2now apply to ONTAP. Both default to off.OntapS3Store::newnow returnsArc<S3Store<_>>. The only callers are in this crate.base64andsha2are no longer used by the store library; I left them inCargo.tomlto keep lockfile churn out of this PR.AI assistance
Claude Code (Opus 5.5) wrote the patch, tests and docs change under my direction. I reviewed the changes, checked the SDK behaviour against the
aws-sdk-s3source, and ran the tests locally.