Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/current/molt/molt-fetch-best-practices.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ toc: true
docs_area: migrate
---

This page describes best practices for using [MOLT Fetch]({% link molt/molt-fetch.md %}) to ensure reliable, secure, and performant data migration to CockroachDB.

## Test and validate

To verify that your connections and configuration work properly, run MOLT Fetch in a staging environment before migrating any data in production. Use a test or development environment that closely resembles production.
Expand Down
2 changes: 2 additions & 0 deletions src/current/molt/molt-fetch-commands-and-flags.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ toc: true
docs_area: migrate
---

This page lists the [MOLT Fetch]({% link molt/molt-fetch.md %}) commands and the flags that you can use to configure a MOLT Fetch command execution.

## Commands

| Command | Usage |
Expand Down
2 changes: 2 additions & 0 deletions src/current/molt/molt-fetch-installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ toc: true
docs_area: migrate
---

This page explains the prequisites for using [MOLT Fetch]({% link molt/molt-fetch.md %}) and then describes how to install it.

## Prerequisites

### Supported databases
Expand Down
2 changes: 2 additions & 0 deletions src/current/molt/molt-fetch-monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ toc: true
docs_area: migrate
---

This page lists the [MOLT Fetch]({% link molt/molt-fetch.md %}) metrics that you can use to observe the progress of a MOLT Fetch command execution.

## Metrics

By default, MOLT Fetch exports [Prometheus](https://prometheus.io/) metrics at `127.0.0.1:3030/metrics`. You can configure this endpoint with the [`--metrics-listen-addr`]({% link molt/molt-fetch-commands-and-flags.md %}#metrics-listen-addr) flag.
Expand Down
2 changes: 2 additions & 0 deletions src/current/molt/molt-fetch-troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ toc: true
docs_area: migrate
---

This page describes common issues that can occur while using [MOLT Fetch]({% link molt/molt-fetch.md %}) and suggests ways to troubleshoot those issues.

<div class="filters filters-big clearfix">
<button class="filter-button" data-scope="postgres">PostgreSQL</button>
<button class="filter-button" data-scope="mysql">MySQL</button>
Expand Down
6 changes: 4 additions & 2 deletions src/current/molt/molt-fetch.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ MOLT Fetch moves data from a source database into CockroachDB as part of a [data

MOLT Fetch uses [`IMPORT INTO`]({% link {{site.current_cloud_version}}/import-into.md %}) or [`COPY FROM`]({% link {{site.current_cloud_version}}/copy.md %}) to move the source data to cloud storage (Google Cloud Storage, Amazon S3, or Azure Blob Storage), a local file server, or local memory. Once the data is exported, MOLT Fetch loads the data into a target CockroachDB database.

You can use MOLT Fetch to migrate data from a PostgreSQL, MySQL, or Oracle source database. Read more about [MOLT Fetch prerequisites]({% link molt/molt-fetch-installation.md %}#prerequisites).

<!-- ## Terminology

- *Shard*: A portion of a table's data exported concurrently during the data export phase. Tables are divided into shards to enable parallel processing. For details, refer to [Table sharding](#shard-tables-for-concurrent-export).
Expand All @@ -29,9 +31,9 @@ In this first phase, MOLT Fetch connects to the source database and exports tabl

- [**Selective data movement**](#select-data-to-migrate): By default, MOLT Fetch moves all data from the --source database to CockroachDB. If instead you want to move a subset of the available data, use the [`--schema-filter`]({% link molt/molt-fetch-commands-and-flags.md %}#schema-filter), [`--table-filter`]({% link molt/molt-fetch-commands-and-flags.md %}#table-filter), and [`--filter-path`]({% link molt/molt-fetch-commands-and-flags.md %}#schema-filter) flags.

- [**Table sharding for concurrent export**](#shard-tables-for-concurrent-export): Multiple tables and **table shards** can be exported simultaneously using [`--table-concurrency`]({% link molt/molt-fetch-commands-and-flags.md %}#table-concurrency) and [`--export-concurrency`]({% link molt/molt-fetch-commands-and-flags.md %}#export-concurrency), with large tables divided into shards for parallel processing.
- [**Table sharding for concurrent export**](#shard-tables-for-concurrent-export): Multiple tables and _table shards_ can be exported simultaneously using [`--table-concurrency`]({% link molt/molt-fetch-commands-and-flags.md %}#table-concurrency) and [`--export-concurrency`]({% link molt/molt-fetch-commands-and-flags.md %}#export-concurrency), with large tables divided into shards for parallel processing.

- [**Load into intermediate storage**](#define-intermediate-storage): Define whether data is written to cloud storage (Amazon S3, Google Cloud Storage, Azure Blob Storage), a local file server, or directly to CockroachDB memory. Intermediate storage enables [continuation after a MOLT Fetch failure](#continue-molt-fetch-after-interruption) by storing **continuation tokens**.
- [**Load into intermediate storage**](#define-intermediate-storage): Define whether data is written to cloud storage (Amazon S3, Google Cloud Storage, Azure Blob Storage), a local file server, or directly to CockroachDB memory. Intermediate storage enables [continuation after a MOLT Fetch failure](#continue-molt-fetch-after-interruption) by storing _continuation tokens_.

### Data import phase

Expand Down
2 changes: 2 additions & 0 deletions src/current/molt/molt-replicator-best-practices.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ toc: true
docs_area: migrate
---

This page describes best practices for using [MOLT Replicator]({% link molt/molt-replicator.md %}) to ensure reliable, secure, and performant data migration to CockroachDB.

## Test and validate

To verify that your connections and configuration work properly, run MOLT Replicator in a staging environment before replicating any data in production. Use a test or development environment that closely resembles production.
Expand Down
2 changes: 2 additions & 0 deletions src/current/molt/molt-replicator-installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ toc: true
docs_area: migrate
---

This page explains the prequisites for using [MOLT Replicator]({% link molt/molt-replicator.md %}) and then describes how to install it.

## Prerequisites

### Supported databases
Expand Down
2 changes: 2 additions & 0 deletions src/current/molt/molt-replicator-troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ toc: true
docs_area: migrate
---

This page describes common issues that can occur while using [MOLT Replicator]({% link molt/molt-replicator.md %}) and suggests ways to troubleshoot those issues.

<div class="filters filters-big clearfix">
<button class="filter-button" data-scope="postgres">PostgreSQL</button>
<button class="filter-button" data-scope="mysql">MySQL</button>
Expand Down
2 changes: 1 addition & 1 deletion src/current/molt/molt-replicator.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ docs_area: migrate

MOLT Replicator continuously replicates changes from a source database to CockroachDB as part of a [database migration]({% link molt/migration-overview.md %}). It supports migrations from a source database to CockroachDB with minimal downtime, and enables backfill from CockroachDB to your source database for failback scenarios to preserve a rollback option during a migration window.

MOLT Replicator consumes change data from PostgreSQL [logical replication](https://www.postgresql.org/docs/current/logical-replication.html) streams, MySQL [GTID-based replication](https://dev.mysql.com/doc/refman/8.0/en/replication-gtids.html), Oracle [LogMiner](https://docs.oracle.com/en/database/oracle/oracle-database/21/sutil/oracle-logminer-utility.html), and [CockroachDB changefeeds]({% link {{ site.current_cloud_version }}/change-data-capture-overview.md %}) (for failback). For details, refer to [How it works](#how-it-works).
MOLT Replicator consumes change data from PostgreSQL [logical replication](https://www.postgresql.org/docs/current/logical-replication.html) streams, MySQL [GTID-based replication](https://dev.mysql.com/doc/refman/8.0/en/replication-gtids.html), Oracle [LogMiner](https://docs.oracle.com/en/database/oracle/oracle-database/21/sutil/oracle-logminer-utility.html), and [CockroachDB changefeeds]({% link {{ site.current_cloud_version }}/change-data-capture-overview.md %}) (for failback). Read more about [MOLT Replicator prerequisites]({% link molt/molt-replicator-installation.md %}#prerequisites).

## Terminology

Expand Down
4 changes: 3 additions & 1 deletion src/current/molt/replicator-flags.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
---
title: Replicator Flags
title: MOLT Replicator Commands and Flags
summary: Flag reference for MOLT Replicator
toc: false
docs_area: migrate
---

This page lists the [MOLT Replicator]({% link molt/molt-replicator.md %}) commands and the flags that you can use to configure a MOLT Replicator command execution.

## Commands

MOLT Replicator provides the following commands:
Expand Down
2 changes: 1 addition & 1 deletion src/current/molt/replicator-metrics.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Replicator Metrics
title: MOLT Replicator Metrics
summary: Learn how to monitor stages of the MOLT Replicator pipeline.
toc: true
docs_area: migrate
Expand Down
Loading