[miniflare] Expose R2 via an S3-compatible API locally - #14280
Merged
Conversation
🦋 Changeset detectedLatest commit: 930b57d The changes in this PR will be included in the next version bump. This PR includes changesets to release 8 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
workers-devprod
requested review from
a team and
penalosa
and removed request for
a team
June 12, 2026 15:52
Contributor
|
Codeowners approval required for this PR:
Show detailed file reviewers
|
@cloudflare/autoconfig
@cloudflare/config
create-cloudflare
@cloudflare/deploy-helpers
@cloudflare/kv-asset-handler
miniflare
@cloudflare/pages-shared
@cloudflare/unenv-preset
@cloudflare/vite-plugin
@cloudflare/vitest-pool-workers
@cloudflare/workers-auth
@cloudflare/workers-editor-shared
@cloudflare/workers-utils
wrangler
commit: |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
petebacondarwin
previously requested changes
Jun 16, 2026
petebacondarwin
left a comment
petebacondarwin
left a comment
Contributor
There was a problem hiding this comment.
@tahmid-23 - thanks for putting this together. It is indeed a lot of change. I think it is much more likely to be reviewed if we do break it up into the initial bug fixes, and then a second PR to add the S3 compatibility. Apart from anything we will need to get R2 product sign off on this as a feature, whereas bug fixes we can land more simply.
tahmid-23
force-pushed
the
r2-local-s3-staged
branch
2 times, most recently
from
June 16, 2026 11:06
3703a8b to
b14551c
Compare
tahmid-23
force-pushed
the
r2-local-s3-staged
branch
from
June 16, 2026 11:34
b14551c to
edefe3b
Compare
5 tasks
Contributor
Author
@petebacondarwin fair enough. I reordered a bit and put the first 8 commits into #14323. |
tahmid-23
force-pushed
the
r2-local-s3-staged
branch
2 times, most recently
from
June 16, 2026 12:40
a6519e6 to
dee1979
Compare
When set, the R2 bucket will be served over a local S3-compatible API during local development, authenticated with the configured AWS SigV4 credentials. This commit adds the config surface and validation; the endpoint itself and the wrangler dev wiring follow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pure refactor: namespace entries flow through verbatim instead of being rebuilt from a fixed field list, so a plugin can carry extra per-entry fields without forking the helper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Buckets configured with s3Credentials are served by a new r2:s3 service at /cdn-cgi/local/r2/s3/<bucket-id>. This commit adds the plugin option, the credential-conflict check, the entry-worker routing, and a stub worker that resolves buckets and answers with S3-style XML errors; the actual S3 protocol follows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…3 endpoint Everything both SigV4 authentication methods (Authorization header and presigned URL query parameters) share: canonical request construction (with S3's single-encoded canonical URI), the signing-key derivation and signature comparison, credential-scope and date parsing with R2's validation order and error messages, and the SignatureDoesNotMatch debug response. verifyRequest() is wired into dispatch but rejects everything with R2's missing-authorization error until the two verifiers land in the following commits. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… S3 endpoint Implements the Authorization-header authentication method on top of the shared verification core: header field parsing, the x-amz-date / date-header fallback, request-time skew bounds, payload-hash verification, and R2's error responses for each failure. Authenticated requests still answer NotImplemented; operations follow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…point Implements the SigV4 query-parameter authentication method on top of the shared verification core: required presigned parameters reported together like R2, X-Amz-Expires bounds (403 ExpiredRequest past expiry, a week at most), the UNSIGNED-PAYLOAD canonical request, and X-Amz-Signature exclusion from the canonical query string. The Authorization header keeps precedence over presigned query parameters. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…'s errors for unimplemented surfaces Adds the operation pipeline: detection yields either a terminal Response or a BoundOperation whose screening rules are applied before its handler runs. Detection reproduces R2's routing for the surfaces its S3 endpoint recognizes but does not implement: object/bucket subresource catch-alls with R2's templated "<name> not implemented" errors (including the GetGetBucketPolicyStatus typo), bucket PUT/DELETE subresource routing, the unsigned bucket-POST presigned-post 501, and a header-screening skeleton (x-amz-security-token rejection). With no operations implemented yet, everything else falls back to a screened NotImplemented operation; each operation group replaces that fallback as its detection lands. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pure refactor: move range parsing, object headers, conditional handling, and range responses out of public.worker.ts into a serveR2Object() shared with the upcoming S3 endpoint, parameterized by endpoint-specific error handlers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CreateMultipartUpload, UploadPart, UploadPartCopy, CompleteMultipartUpload, and AbortMultipartUpload, built on the binding's resumeMultipartUpload(). Binding errors are mapped back onto R2's S3 error responses by their v4 codes (workerd only exposes them via the error message), and the simulator's internal-error responses for unknown upload ids are mapped onto NoSuchUpload. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The account-level route verifies the request against each configured credential set (preferring the most specific auth error) and lists the buckets matching the presented credentials. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…the local S3 endpoint Pass the configured credentials through to miniflare's r2Buckets option for local (non-remote) buckets, enabling the S3-compatible endpoint during wrangler dev. With the endpoint now fully wired up, this also adds the miniflare changeset, documents the known gaps vs real R2 in the S3 worker's module header, and aligns the lockfile with the new miniflare devDependencies. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tahmid-23
force-pushed
the
r2-local-s3-staged
branch
from
June 29, 2026 16:02
dee1979 to
b055770
Compare
Contributor
|
Codeowners approval required for this PR:
Show detailed file reviewers
|
Contributor
Author
|
rebased on top of the merged fix commits. doesn't do much to reduce the diff though... |
helloimalastair
left a comment
helloimalastair
left a comment
Contributor
There was a problem hiding this comment.
A few small nits here, but otherwise approved by R2 Team Member.
cc @emily-shen
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
Codeowners approval required for this PR:
Show detailed file reviewers
|
Contributor
Author
|
nits addressed, merged main to fix conflicts |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
emily-shen
reviewed
Jul 24, 2026
emily-shen
left a comment
emily-shen
left a comment
Contributor
There was a problem hiding this comment.
Sorry about the long delay getting back to you! This is very much appreciated 🧡
I'm basically ready to approve this, we just happen to be currently working out what we want dev-only resource config to look like for another resource, and i'd like experimental_local_s3_credentials to be consistent with that.
In terms of next steps:
- docs - this should definitely have docs, probably under a 'local development' page for R2, which is long overdue anyway. If you could submit a PR, that'd be great - I'm very happy to help as well.
- once this is out in a release, i'll post a comment on the original feature request to get people testing, and tag you with credit.
emily-shen
reviewed
Jul 24, 2026
Contributor
There was a problem hiding this comment.
would you also be able to add an e2e test in wrangler (e2e/dev.test.ts) and vite (either in packages/vite-plugin-cloudflare/playground/bindings or a new fixture)? specifically it would be good to have a regression test to ensure that the sigv4 verification is interacting correctly with the routing we do in our different dev servers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
Codeowners approval required for this PR:
Show detailed file reviewers |
Regression tests that SigV4 verification interacts correctly with the request routing in wrangler dev (e2e/dev.test.ts) and the Vite plugin (playground/bindings): header-auth round-trip, presigned URL query auth, and rejection of bad signatures through each dev server. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2 tasks
Contributor
Author
|
@emily-shen added test + cloudflare/cloudflare-docs#32319 |
emily-shen
reviewed
Jul 27, 2026
emily-shen
left a comment
emily-shen
left a comment
Contributor
There was a problem hiding this comment.
just the config change then lgtm :)
Renames experimental_local_s3_credentials to local_dev.experimental_s3_credentials per review, aligning with the direction for dev-only resource config. Also validates that local_dev is an object and rejects unknown fields within it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
emily-shen
approved these changes
Jul 27, 2026
workers-devprod
approved these changes
Jul 27, 2026
workers-devprod
left a comment
workers-devprod
left a comment
Contributor
There was a problem hiding this comment.
Codeowners reviews satisfied
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is based off of #14119, which laid the groundwork.
Motivation
R2 provides an S3 compatibility layer. This exposes R2 operations via S3. However, this is currently not replicated in wrangler.
This feature is especially useful for presigned requests, and in particular presigned uploads. Presigned uploads allows a user to directly upload a file to R2, rather than go indirectly through a worker (in production). However, wrangler does not currently support any S3 compatible API, meaning that using presigned uploads in production can't be tested locally (without making your own worker to implement this exact purpose). Otherwise, I do not believe an S3-compatible local API is strictly needed (although I could be mistaken).
Hence, the primary motivation for adding this S3-compatible local API is pre-signed uploads. However, if I'm going to add presigned upload support, I figured I ought to just support S3 to the maximum extent possible, since presigned uploads are fundamentally an S3 concept.
Regardless, having the S3-compatible API locally makes testing easier if using an S3 client is preferred.
Implementation
I realize that this is a sizable change. I did my best effort to split my commits into small chunks, so that each commit is realistically reviewable. The commits build upon each other sequentially / do not revert prior work (up to Add changesets for the local R2 S3 endpoint and public-bucket fixes, e.g. before any review feedback commits).
Public Worker Fixes
The first few commits are series of patches to the existing local public worker. These are minor bugs / inconsistencies with cloudflare's public R2 endpoints. (I wanted to drive parity here, since I didn't feel that the bugs necessarily warranted their own PR. The functionality of these fixes is also equally important for correctness in the S3 API.)
Configuration
We expose the following configuration field in
wrangler.jsonc:{ "r2_buckets": [ { "binding": "BUCKET", "bucket_name": "my-bucket", "experimental_local_s3_credentials": { "accessKeyId": "local-access-key-id", "secretAccessKey": "local-secret-access-key", }, }, ], }If any bucket is configured with
experimental_local_s3_credentials, it will launch the S3 worker atcdn-cgi/local/r2/s3, and use the credentials to authorize requests to the local S3 API.Authorization
We implement AWS's SigV4 signing algorithm on the server-side. This is implemented for both Authorization header authorization (for using the standard s3 client), and for presigned query parameter authorization (for presigned requests). These are verified by using the standard
@smithy/signature-v4package in tests.Operations
The cloudflare docs on S3 compatibility describes which operations are fully supported. We implement these to the maximum extent possible.
However, some features are not implementable in local development. For example, "SSE-C" cannot be supported, because miniflare does not internally expose any API to forward the SSE-C parameter to workerd R2.
index.worker.tscontains a comment describing some of the limitations.I used Claude to determine the error messages and conditions against my public R2-S3 endpoint. I tried to maintain parity with production R2-S3 as much as possible (error message formatting, error codes, order of checks, etc.)
A picture of a cute animal (not mandatory, but encouraged)