Skip to content

Changes to support separate bucket per pg cluster (#1209) #1239

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ajaybhat
Copy link

@ajaybhat ajaybhat commented Dec 3, 2020

Changes Done

  • WAL_S3_BUCKET used in pg configmap will result in the cluster using that bucket for streaming/pushing WAL archives. In case it's not set, the default bucket wal_s3_bucket from operator configmap will be used. This bucket should be accessible by the user AWS_ACCESS_KEY_ID in pg configmap.
  • LOGICAL_BACKUP_S3_BUCKET used in pg configmap will result in the cluster using that bucket for pushing logical backups. In case its not set, the default bucket logical_backup_s3_bucket from operator configmap will be used. This bucket should be accessible by the user logical_backup_s3_access_key_id in operator configmap. Using a custom user for a logical backup bucket is not included in this PR.

Ideally, we will always use the same bucket for WAL and logical backups.

Tests performed

Scenario Result
Don't set WAL_S3_BUCKET or LOGICAL_BACKUP_S3_BUCKET. Create primary Default bucket from operator used. WAL and backups pushed to default
Don't set WAL_S3_BUCKET or LOGICAL_BACKUP_S3_BUCKET. Create standby Default bucket from operator used. Only backups pushed to default
Set custom WAL_S3_BUCKET and LOGICAL_BACKUP_S3_BUCKET. Create primary Custom bucket from configmap used. WAL and backups pushed to custom
Set custom WAL_S3_BUCKET and LOGICAL_BACKUP_S3_BUCKET. Create standby to stream from default bucket Custom bucket from configmap used. The cluster starts streaming from the primary in a different bucket. Only backups pushed to custom bucket
Set custom WAL_S3_BUCKET and don't set LOGICAL_BACKUP_S3_BUCKET. Create primary Default bucket from operator used to push backups and custom bucket from configmap used to push WALs
Set custom WAL_S3_BUCKET and LOGICAL_BACKUP_S3_BUCKET. Clone from custom bucket Cluster cloned from the custom bucket. Custom bucket from configmap used. WAL and backups pushed to custom

Co-authored by: Raja Sekhar <rajasekhar@vmware.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant