Skip to content

moiot/gravity

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Gravity 简体中文


Build Status

2.0 Product

Gravity is used to replicate data between various inputs and outputs (databases, message queues).

It is designed to be a customizable data replication tool that:

  • Supports multiple sources and destinations

  • Supports Kubernetes-based replication cluster

Use Cases

  • Data Bus: Use Change Data Capture (MySQL binlog, MongoDB Oplog) and batch table scan to publish data to message queue like Kafka.
  • Unidirectional data replication: Replicates data from one MySQL cluster to another MySQL cluster.
  • Bidirectional data replication: Replicates data between two MySQL clusters bidirectionally.
  • Synchronization of shards to the merged table: Synchronizes MySQL sharded tables to the merged table. You can specify the corresponding relationship between the source table and the target table.
  • Online data mutation: Data can be changed during the replication. For example, rename the column, encrypt/decrypt data columns.

Features

Input support

Input Status
MySQL Binlog
MySQL Scan
Mongo Oplog
Mongo Scan
TiDB Binlog Doing
PostgreSQL WAL Doing

output support

Output Status
MySQL/TiDB
Kafka
Elasticsearch Beta

Data mutation support

Gravity supports the following data mutations:

  • Ignore specific data
  • Renaming columns
  • Deleting columns

Documentation


Special thanks to @dantin, @LiangShang, and @liwink for the early support to this project, and thanks @CaitinChen for the initial translation of the documentation.