HADOOP-17817. Throw an exception if S3 client-side encryption is enabled on S3Guard enabled bucket #3239
Conversation
…led on S3Guard enabled bucket
|
CC: @steveloughran |
| @@ -539,6 +539,9 @@ public void initialize(URI name, Configuration originalConf) | |||
| if (hasMetadataStore()) { | |||
| LOG.debug("Using metadata store {}, authoritative store={}, authoritative path={}", | |||
| getMetadataStore(), allowAuthoritativeMetadataStore, allowAuthoritativePaths); | |||
| if (isCSEEnabled) { | |||
There was a problem hiding this comment.
Initialize is allowed to throw an IOE; our code tends to expect it.
Can you throw a PathIOE(uri, "S3-CSE cannot be used with S3Guard")
i.e give bucket and problem, but don't propose a solution...leave that to the docs as disabling s3guard is probably the better one, depending on what the user wants
|
💔 -1 overall
This message was automatically generated. |
|
Seems like, I've broken tests for S3-CSE ON because alot of S3Guard tests don't require your bucket to be s3guard enabled, and force the metastore to be dynamoDB. My lapse on testing for S3-CSE ON and S3-Guard OFF. I think we should've skipped the S3Guard tests for S3-CSE anyways, so I'll skip all of them in a follow-up PR. The failure is valid, but still, we should skip, what do you think, @steveloughran? |
|
hmm. Yes, that'll be an interesting problem. Either test setup() checks for CSE on and skips the s3guard-enabled tests, or we catch the raised PathIOE and covert that to the skip call. That strategy might work well everywhere, including all contract tests. Also: did you forget to run the tests? or is it just your test setup isn't S3-CSE? This is where we need broader test configuration coverage, don't we? |
|
I ran the tests in S3-CSE ON S3Guard ON, S3-CSE OFF S3-Guard ON, and S3-CSE OFF S3-Guard OFF. More of a mistake that I thought I had run the S3-CSE ON and S3-Guard OFF test suite. |
…d enabled (apache#3239) S3A S3Guard tests to skip if S3-CSE are enabled (apache#3263) Follow on to * HADOOP-13887. Encrypt S3A data client-side with AWS SDK (S3-CSE) If the S3A bucket is set up to use S3-CSE encryption, all tests which turn on S3Guard are skipped, so they don't raise any exceptions about incompatible configurations. Contributed by Mehakmeet Singh
…d enabled (#3239) S3A S3Guard tests to skip if S3-CSE are enabled (#3263) Follow on to * HADOOP-13887. Encrypt S3A data client-side with AWS SDK (S3-CSE) If the S3A bucket is set up to use S3-CSE encryption, all tests which turn on S3Guard are skipped, so they don't raise any exceptions about incompatible configurations. Contributed by Mehakmeet Singh Change-Id: I9f4188109b56a1f4e5a31fae265d980c5795db1e
…ache#3239) Contributed by Mehakmeet Singh
Region: ap-south-1.
All tests initializing S3AFS failed.
Follow up of #2706.