-
Notifications
You must be signed in to change notification settings - Fork 8.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HADOOP-17198. Support S3 Access Points #3958
HADOOP-17198. Support S3 Access Points #3958
Conversation
Add support for S3 Access Points. This provides extra security as it ensures applications are not working with buckets belong to third parties. To bind a bucket to an access point, set the access point (ap) ARN, which must be done for each specific bucket, using the pattern fs.s3a.bucket.$BUCKET.accesspoint.arn = ARN * The global/bucket option `fs.s3a.accesspoint.required` to mandate that buckets must declare their access point. * This is not compatible with S3Guard. Consult the documentation for further details. Contributed by Bogdan Stolojan (this commit contains the changes to TestArnResource from HADOOP-18068, "upgrade AWS SDK to 1.12.132" so that it works with the later SDK.) Change-Id: I3fac213e52ca6ec1c813effb8496c353964b8e1b
…he#3516) Follow-on to HADOOP-17198. Support S3 Access Points Contributed by Bogdan Stolojan
this is @bogthe's work, I've just dealt with the merge conflict caused by the s3guard cut. testing s3 london. assuming it and the yetus tests are happy, i will do the merge |
🎊 +1 overall
This message was automatically generated. |
thanks for this! |
hmm
sdk merge pain. the 3.3.2 patch is actually easier here |
This patch makes this AP feature independent to SDK upgrade. #3902 |
…lation (apache#3902) Part of HADOOP-17198. Support S3 Access Points. HADOOP-18068. "upgrade AWS SDK to 1.12.132" broke the access point endpoint translation. Correct endpoints should start with "s3-accesspoint.", after SDK upgrade they start with "s3.accesspoint-" which messes up tests + region detection by the SDK. Contributed by Bogdan Stolojan
confirmed; applying that change to this branch fixes it all. i'm rerunning the test suites for safety, but then i'm going to merge in this branch as the three separate changes. Something |
ok, reran all tests against s3 london, all good . merging locally as a chain of commits, rather than squashing in through the github web interface |
Thanks @steveloughran ! can this be cherry-picked cleanly into |
awesome work @steveloughran thank you! |
🎊 +1 overall
This message was automatically generated. |
HADOOP-17198. Support S3 Access Points (#3260)
Add support for S3 Access Points. This provides extra security as it
ensures applications are not working with buckets belong to third parties.
To bind a bucket to an access point, set the access point (ap) ARN,
which must be done for each specific bucket, using the pattern
fs.s3a.bucket.$BUCKET.accesspoint.arn = ARN
fs.s3a.accesspoint.required
tomandate that buckets must declare their access point.
Consult the documentation for further details.
Contributed by Bogdan Stolojan
(this commit contains the changes to TestArnResource from HADOOP-18068,
"upgrade AWS SDK to 1.12.132" so that it works with the later SDK.)
For code changes:
LICENSE
,LICENSE-binary
,NOTICE-binary
files?