Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ticdc: update ticdc-debezium.md #19544

Open
wants to merge 31 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
de39cfc
wip
wk989898 Dec 2, 2024
8c957e5
Update ticdc-debezium.md
wk989898 Dec 6, 2024
a0dc693
Update ticdc-debezium.md
wk989898 Dec 6, 2024
793e485
Update ticdc-debezium.md
wk989898 Dec 6, 2024
66f2dde
Update ticdc-debezium.md
wk989898 Dec 6, 2024
b9c4f4b
Update ticdc-debezium.md
wk989898 Dec 6, 2024
6da4609
Update ticdc-debezium.md
wk989898 Dec 6, 2024
afc151e
Update ticdc-debezium.md
wk989898 Dec 9, 2024
19aa23c
Update ticdc-debezium.md
wk989898 Dec 9, 2024
61b0b2a
Update ticdc-debezium.md
wk989898 Dec 9, 2024
996d8f9
Update ticdc-debezium.md
wk989898 Dec 12, 2024
852aa42
ticdc: replace each hard Tab with four spaces
lilin90 Jan 6, 2025
abbe96d
ticdc: update format
lilin90 Jan 6, 2025
bc9013d
ticdc: avoid manual line breaks
lilin90 Jan 6, 2025
e6311da
ticdc: fix unclosed tags
lilin90 Jan 6, 2025
5c2a154
Update ticdc-debezium.md
wk989898 Jan 10, 2025
b4fbf1d
Update ticdc-debezium.md
wk989898 Jan 13, 2025
70688e0
Apply suggestions from code review
wk989898 Jan 24, 2025
da11ba4
Update ticdc-debezium.md
wk989898 Jan 24, 2025
0553e52
Update wording for consistency
lilin90 Feb 5, 2025
a3aa368
Apply suggestions from code review
wk989898 Feb 6, 2025
1e15afb
Apply suggestions from code review
wk989898 Feb 6, 2025
beb5c8b
Update according to comments on Chinese PR
lilin90 Feb 6, 2025
d1b79b4
Update wording for consistency
lilin90 Feb 6, 2025
3d3e58d
ticdc: improve format
lilin90 Feb 6, 2025
ab61ba3
Apply suggestions from code review
lilin90 Feb 7, 2025
35990d9
Remove extra spaces
lilin90 Feb 7, 2025
de26af5
Apply suggestions from code review
lilin90 Feb 7, 2025
9df3fce
Remove an extra space
lilin90 Feb 7, 2025
8954f1a
Apply suggestions from code review
lilin90 Feb 8, 2025
fefe294
Update format
lilin90 Feb 8, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
wip
  • Loading branch information
wk989898 authored Dec 2, 2024
commit de39cfc2543482789ae92a1f7fd2445530d88c02
9 changes: 7 additions & 2 deletions ticdc/ticdc-debezium.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ summary: Learn the concept of the TiCDC Debezium Protocol and how to use it.

# TiCDC Debezium Protocol

[Debezium](https://debezium.io/) is a tool for capturing database changes. It converts each captured database change into a message called an "event" and sends these events to Kafka. Starting from v8.0.0, TiCDC supports sending TiDB changes to Kafka using a Debezium style output format, simplifying migration from MySQL databases for users who had previously been using Debezium's MySQL integration.
[Debezium](https://debezium.io/) is a tool for capturing database changes. It converts each captured database change into a message called an "event" and sends these events to Kafka. Starting from v8.0.0, TiCDC supports sending TiDB changes to Kafka using a Debezium style output format, simplifying migration from MySQL databases for users who had previously been using Debezium's MySQL integration. Starting from v8.6, TiCDC supports DDL events and watermark events.

## Use the Debezium message format

Expand All @@ -27,11 +27,13 @@ In addition, the original Debezium format does not include important fields such

This section describes the format definition of the DML event output in the Debezium format.

### DDL event

### DML event

TiCDC encodes a DML event into a Kafka message, with both the key and value encoded in the Debezium format.

### Key format
#### Key format

```json
{
Expand Down Expand Up @@ -164,6 +166,9 @@ The key fields of the preceding JSON data are explained as follows:
| schema.optional| Boolean | Indicates whether the field is optional. When it is `true`, the field is optional. |
| schema.type | String | The data type of the field. |

### WATERMARK
wk989898 marked this conversation as resolved.
Show resolved Hide resolved


### Data type mapping

The data format mapping in the TiCDC Debezium message basically follows the [Debezium data type mapping rules](https://debezium.io/documentation/reference/2.4/connectors/mysql.html#mysql-data-types), which is generally consistent with the native message of the Debezium Connector for MySQL. However, for some data types, the following differences exist between TiCDC Debezium and Debezium Connector messages:
Expand Down