Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable archival config per domain #1351

Merged
merged 26 commits into from
Dec 21, 2018

Conversation

andrewjdawson2016
Copy link
Contributor

@andrewjdawson2016 andrewjdawson2016 commented Dec 20, 2018

Diff Contains The Following

  • Update IDL to include fields to operate on domain archival config
  • Update schema for SQL and Cassandra to keep archival config info
  • Do all plumbing between IDL and database
  • Introduce deployment group into cluster config
  • Add persistence level unit tests
  • Add workflowHandler level unit tests
  • Add admin cli to operate over archival config until we are ready to generally release archival

Concepts to Understand

  • Once archival is enabled a bucket name is always set and can never be changed.
  • There are three states archival can be in never_enabled, disabled and enabled. Never enabled means that archival has never been turned on for that domain. Enabled means archival is currently on for that domain and disabled means archival is currently off for domain.
  • There are only three allowable archival status changes never_enabled->enabled, enabled->disabled and disabled->enabled.
  • When archival is turned on for the first time a bucket name is always put in the database (either a custom user provided bucket or a default cadence bucket).
  • After a bucket name is stored in the database it is never unset, even when archival is disabled.
  • In order to update configuration on bucket such as retention time or owner the blobstore must be accessed directly.
  • There are only three valid states database can be in: {never_enabled&"", enabled&"bucket_name", disabled&"bucket_name"}.
  • The default state of the database is never_enabled and empty bucket name (all domains will start with these values). A domain can be updated to enable archival. Upon first enabling archival an already existing domain gets a one time change to optionally also set a custom bucket name.

30: optional string archivalBucketName
40: optional i32 archivalRetentionPeriodInDays
50: optional ArchivalStatus archivalStatus
60: optional string archivalBucketOwner
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need to have a separate owner information for the bucket as part of domain configuration? This is more like bucket metadata and should belong to blob store.

@andrewjdawson2016 andrewjdawson2016 merged commit 0fe330b into uber:master Dec 21, 2018
@andrewjdawson2016 andrewjdawson2016 deleted the ArcCfg branch January 5, 2019 20:13
@dmetzgar dmetzgar added this to the Archival milestone Jan 11, 2019
@dmetzgar dmetzgar added the archival Workflow history archival label May 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
archival Workflow history archival
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants