Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support S3 bucket ownership validation on the S3 sink #4468

Closed
dlvenable opened this issue Apr 26, 2024 · 1 comment · Fixed by #4504
Closed

Support S3 bucket ownership validation on the S3 sink #4468

dlvenable opened this issue Apr 26, 2024 · 1 comment · Fixed by #4504
Labels
enhancement New feature or request
Milestone

Comments

@dlvenable
Copy link
Member

Is your feature request related to a problem? Please describe.

The s3 source supports S3's bucket ownership verification to protect against reading from buckets in unexpected accounts.

The s3 sink does not have this feature.

Describe the solution you'd like

Provide the same configurations for S3 bucket ownership as are provided in the s3 source. Use those to define the ExpectedBucketOwner parameter when writing to S3.

sink:
  - s3:
      default_bucket_owner: 000000000000
      bucket_owners:
        my-bucket-01: 123456789012
        my-bucket-02: 999999999999

Conceptual PutObjectRequest:

PutObjectRequest.builder().bucket(defaultBucket).key(objectKey).expectedBucketOwner(buckerOwner).build()

Additionally, this check should occur based on either of those fields being set. If they are not set, then there is no check. This is the current default and this will not break anything. As a result, I don't see any need to have a disable flag.

Additional context

Issue to add these configurations to the s3 source: #2012.

Original PR adding the check to the s3 source: #1526

@dlvenable dlvenable added untriaged enhancement New feature or request and removed untriaged labels Apr 26, 2024
@dlvenable dlvenable added this to the v2.8 milestone Apr 30, 2024
@Utkarsh-Aga
Copy link
Contributor

Hello @dlvenable,
Wanted to check, if someone is actively working on this, or not. If not, can I work on this ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Development

Successfully merging a pull request may close this issue.

2 participants