Skip to content

Commit

Permalink
fix issues of weekly meeting (#6048)
Browse files Browse the repository at this point in the history
  • Loading branch information
sunzhaoyang authored Jul 29, 2021
1 parent 1327ee8 commit 6e10f3f
Show file tree
Hide file tree
Showing 24 changed files with 110 additions and 79 deletions.
33 changes: 16 additions & 17 deletions TOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,32 +42,31 @@
+ [Test TiDB Using TPC-C](/benchmark/benchmark-tidb-using-tpcc.md)
+ Migrate
+ [Overview](/migration-overview.md)
+ Migrate from MySQL
+ [Overview](/data-migration/mysql/overview.md)
+ [Origin MySQL ≥ 1TB](/data-migration/mysql/huge-data.md)
+ [Origin MySQL < 1TB](/data-migration/mysql/small-data.md)
+ [Incremental migration](/data-migration/mysql/increment.md)
+ Migrate from Amazon Aurora
+ [Overview](/data-migration/aurora/overview.md)
+ From MySQL To TiDB
+ [Overview](/data-migration/mysql/migrate-mysql-overview.md)
+ [MySQL ≥ 1TB](/data-migration/mysql/huge-data.md)
+ [MySQL < 1TB](/data-migration/mysql/small-data.md)
+ [Incremental migration](/data-migration/mysql/increment-mysql.md)
+ From Amazon Aurora To TiDB
+ [Overview](/data-migration/aurora/migrate-aurora-overview.md)
+ [From Aurora snapshot](/data-migration/aurora/from-snapshot.md)
+ [Incremental migration](/data-migration/aurora/increment.md)
+ Migrate from Files
+ [Overview](/data-migration/file/overview.md)
+ [Incremental migration](/data-migration/aurora/increment-aurora.md)
+ From Files To TiDB
+ [Overview](/data-migration/file/file-migration-overview.md)
+ [CSV files](/data-migration/file/csv-tidb.md)
+ [SQL files](/data-migration/file/sql-tidb.md)
+ Migrate from Cloud
+ [Overview](/data-migration/cloud/overview.md)
+ From Cloud To TiDB
+ [Overview](/data-migration/cloud/migrate-cloud-overview.md)
+ [Amazon](/data-migration/cloud/aws-tidb.md)
+ [Google](/data-migration/cloud/google-tidb.md)
+ Advanced Migration
+ [Overview](/data-migration/advanced-migration/overview.md)
+ From TiDB To TiDB
+ [Overview](/data-migration/tidb-tidb.md)
+ Advanced Migration To TiDB
+ [Overview](/data-migration/advanced-migration/advanced-migration-overview.md)
+ [Filter binlog event](/data-migration/advanced-migration/binlog-filter.md)
+ [Work with GH-ost and PT-osc](/data-migration/advanced-migration/ghost-ptosc.md)
+ [Merge multiple tables and databases](/data-migration/advanced-migration/merge-db-table.md)
+ [Multiple tables downstream](/data-migration/advanced-migration/more-column.md)
+ Case Example
+ [Migrate from Aurora to TiDB](/data-migration/case/migrate-aurora-tidb.md)
+ [Merge multi tables and databases into one](/data-migration/case/migrate-multi-table.md)

+ Maintain
+ Upgrade
Expand Down
Empty file removed TODO
Empty file.
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,6 @@ These functions only applicable if the downstream is TiDB, and do not applicable
- [Merge multi tables and databases](/data-migration/advanced-migration/merge-db-table.md)
- [Multiple tables downstream](/data-migration/advanced-migration/more-column.md)

To illustrate more vividly, we have prepared actual cases for your reference,please refer to the following topics.

***

## Topics

- [Merge multi tables and databases into one](/data-migration/case/migrate-multi-table.md)
3 changes: 1 addition & 2 deletions data-migration/aurora/from-snapshot.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ Currently, TiDB Lightning supports:
Because the snapshot data exported from Aurora to S3 does not contain the SQL statement file used to create database tables, you need to manually export and import the table creation statements corresponding to the database tables into TiDB. You can use Dumpling and TiDB Lightning to create all table schemas:

- [Export snapshots of Aurora to S3](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_ExportSnapshot.html)
- [Export table schema using dumpling](/data-migration/todo.md)
- [Deploy Lighting using TiUP](/data-migration/todo.md)
- [Deploy Dumping using TiUP](/data-migration/todo.md)

Expand Down Expand Up @@ -132,6 +131,6 @@ When the import operation is started, view the progress by the following two way

## Helpful Topics

- [Incrementally synchronize data From Aurora MySQL to TiDB](/data-migration/aurora/increment.md)
- [Incrementally synchronize data From Aurora MySQL to TiDB](/data-migration/aurora/increment-aurora.md)

- [Lighting Administration Guide](https://docs.pingcap.com/tidb/stable/tidb-lightning-overview)
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
title:
summary:
---

# Incrementally synchronize data From Aurora MySQL to TiDB

[TiDB Data Migration](https://github.com/pingcap/dm) (DM) is an integrated data migration task management platform, which supports the full data migration and the incremental data replication from MySQL-compatible databases (such as MySQL, MariaDB, and Aurora MySQL) into TiDB. It can help to reduce the operation cost of data migration and simplify the troubleshooting process.
Expand Down
23 changes: 23 additions & 0 deletions data-migration/aurora/migrate-aurora-overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
title: Migrate from Amazon Aurora MySQL to TiDB
summary: Learn how to migrate full data from Amazon Aurora MySQL to TiDB.
---

# Overview of migration from Aurora to TiDB

This document introduces how to migrate full data from Amazon Aurora MySQL to TiDB.

[Dumpling](https://github.com/pingcap/dumpling) exports data stored in TiDB/MySQL as SQL or CSV data files and can be used to make a logical full backup or export.

[Lightning](/tidb-lightning/tidb-lightning-overview.md) is a tool used for fast full import of large amounts of data into a TiDB cluster.

[Data Migration](https://github.com/pingcap/dm) (DM) is an data migration task management platform, supports the full data migration and incremental data replication into TiDB.

Using Aurora's existing snapshot mechanism is the most convenient way, but beacuse [DM](https://github.com/pingcap/dm) does not yet support the parquet format, so we need to use [Lightning](https://docs.pingcap.com/tidb/stable/tidb-lightning-overview) for full data import first, and then use [DM](https://github.com/pingcap/dm) for incremental data synchronization.

***

## Topics

- [Migrate from Aurora snapshot to TiDB](/data-migration/aurora/from-snapshot.md)
- [Incrementally synchronize data to TiDB](/data-migration/aurora/increment-aurora.md)
21 changes: 0 additions & 21 deletions data-migration/aurora/overview.md

This file was deleted.

Empty file.
Empty file.
4 changes: 4 additions & 0 deletions data-migration/cloud/aws-tidb.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
---
title:
summary:
---
# How to migrate from AWS to TiDB

TODO: quick instruction
Expand Down
4 changes: 4 additions & 0 deletions data-migration/cloud/google-tidb.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
---
title:
summary:
---
# How to migrate from Google to TiDB

TODO: quick instruction
Expand Down
4 changes: 4 additions & 0 deletions data-migration/cloud/migrate-cloud-overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
title:
summary:
---
Empty file removed data-migration/cloud/overview.md
Empty file.
13 changes: 0 additions & 13 deletions data-migration/data-migration.md

This file was deleted.

4 changes: 4 additions & 0 deletions data-migration/file/csv-tidb.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
---
title:
summary:
---
# How to migrate from CSV file to TiDB

TODO: quick instruction
Expand Down
4 changes: 4 additions & 0 deletions data-migration/file/file-migration-overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
title:
summary:
---
Empty file removed data-migration/file/overview.md
Empty file.
4 changes: 4 additions & 0 deletions data-migration/file/sql-tidb.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
---
title:
summary:
---
# How to migrate from SQL file to TiDB

TODO: quick instruction
Expand Down
7 changes: 5 additions & 2 deletions data-migration/mysql/huge-data.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
---
title:
summary:
---
# Migrating from ≥1TB MySQL to TiDB

TODO: quick instruction of lighting

## Before starting the migration, complete the following tasks

- [Export snapshots of Aurora to S3](/data-migration/todo.md)
- [Deploy lighting using tiup](/data-migration/todo.md)
- [Deploy Lighting using tiup](/data-migration/todo.md)

TODO: operations of migration

Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,15 @@
---
title:
summary:
---

# Migrating from < 1TB MySQL to TiDB

TODO: quick instruction of DM

## Before starting the migration, complete the following tasks

- [Deploy DM using tiup](/data-migration/todo.md)
- [Deploy DM using TiUP](/data-migration/todo.md)

TODO: operations of migration

Expand Down
22 changes: 22 additions & 0 deletions data-migration/mysql/migrate-mysql-overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
title: Migrate from MySQL to TiDB
summary: Learn how to migrate full data from MySQL to TiDB.
---

# Overview of migration from MySQL to TiDB

This document introduces how to migrate full data from MySQL to TiDB.

[Dumpling](https://github.com/pingcap/dumpling) exports data stored in TiDB/MySQL as SQL or CSV data files and can be used to make a logical full backup or export.

[Lightning](/tidb-lightning/tidb-lightning-overview.md) is a tool used for fast full import of large amounts of data into a TiDB cluster.

[Data Migration](https://github.com/pingcap/dm) (DM) is an data migration task management platform, supports the full data migration and incremental data replication into TiDB.

It is recommended to use [Dumpling](https://github.com/pingcap/dumpling) and [Lightning](/tidb-lightning/tidb-lightning-overview.md) in large data import scenarios, [DM](https://github.com/pingcap/dm) in small and incremental data synchronization scenarios because of the data import speed of DM is not fast.

## Topics

- [MySQL ≥ 1TB](/data-migration/mysql/huge-data.md)
- [MySQL < 1TB](/data-migration/mysql/small-data.md)
- [Incrementally synchronize data to TiDB](/data-migration/mysql/increment-mysql.md)
19 changes: 0 additions & 19 deletions data-migration/mysql/overview.md

This file was deleted.

4 changes: 4 additions & 0 deletions data-migration/mysql/small-data.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
---
title:
summary:
---
# Incrementally synchronize data from MySQL to TiDB

TODO: quick instruction of DM
Expand Down
4 changes: 4 additions & 0 deletions data-migration/tidb-tidb.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
title:
summary:
---

0 comments on commit 6e10f3f

Please sign in to comment.