Skip to content

Transactional, distributed backups #2620

Open
@mbautin

Description

@mbautin

Jira Link: DB-2323
This is a master issue tracking transactional, distributed, point-in-time backups. This feature should enable the following:

  • Take a transactional backup of an entire database, or multiple tables
  • It should include all transactions as of some timestamp
  • A transaction should either be fully included or excluded (depending on the timestamp)
  • Each node would keep a backup of the subset of the total data
  • A snapshot from one cluster can be restored to another cluster possibly with a different number of nodes

NOTE: A backup refers to an in-cluster snapshot which can subsequently get copied out to an external location.

Prerequisites

✅ Design Doc: distributed backups and restore

Phase 1 - non-transactional YCQL tables

✅ Enable instantaneous checkpointing of DocDB data files on each tablet
✅ Ability to initiate backup of one table from YB-Master
✅ Ability to restore one table from a backup (into the same or remote cluster)
✅ Include schemas for YCQL backups

Yugabyte Platform enhancements

✅ [Platform] Backup an entire keyspace in YCQL
✅ [Platform] Nightly backups in a schedule

Phase 2 - Transactional backups - YSQL and YCQL

✅ Checkpoint an entire table at a consistent point in time #1032
✅ Consistent backup of YSQL database (target v2.2) #1139
⬜️ Use backup / restore roles for doing backup / restore #4798
✅ Backup individual tables along with indexes for YCQL
✅ Backup entire database/schema for YSQL

Phase 3 - PITR, incremental backups, cluster backup

⬜️ Ability to perform point in time restores #1820
⬜️ Incremental backups
⬜️ Backup entire YugabyteDB cluster: This will backup all databases as well as roles, permissions and other cluster metadata

Usability improvements

⬜️ Document backups and restore for DB and platform #2349 #2757 #2926 #3241
⬜️ Enable YSQL backups through ysqlsh shell
⬜️ Ability to perform a cluster-wide YCQL backup using ybadmin or ycqlsh #2350

Yugabyte Platform enhancements

⬜️ [Platform] Purge older backups automatically from backup targets

Analytics

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area/ysqlYugabyte SQL (YSQL)kind/enhancementThis is an enhancement of an existing featurepriority/mediumMedium priority issueroadmap-tracking-issueThis issue tracks a major roadmap item, and usually appears in the roadmap list.

Type

No type

Projects

  • Status

    No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions