Skip to content

[Bug]: when bandwidth chaos to cn ,cdc downstream table with fewer rows than upstream #22718

@heni02

Description

@heni02

Is there an existing issue for the same bug?

  • I have checked the existing issues.

Branch Name

3.0-dev

Commit ID

3541760

Other Environment Information

- Hardware parameters:
- OS type:
- Others:

Actual Behavior

测试场景:1个cn 故障注入 network bandwidth 类型,同步cdc 测试,上游持续insert 数据,测试 1 个小时后验证上下游表行数不一致

Image

cdc 前端监控:
https://shanghai.idc.matrixorigin.cn:30001/d/ce3ttwv605r0gb/frontend-metrics?orgId=1&var-interval=1m&var-namespace=mo-benchmark-moi&var-pod=All&from=1761814096291&to=1761821296291
Image

cdc 任务状态,报错:internal error: CDC-MySQLSink retry exceed retryTimes or retryDuration
Image

mo log:
https://shanghai.idc.matrixorigin.cn:30001/explore?panes=%7B%221vB%22:%7B%22datasource%22:%22loki%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22%7Bnamespace%3D%5C%22mo-benchmark-moi%5C%22%7D%20%7C%3D%20%60%60%22,%22queryType%22:%22range%22,%22datasource%22:%7B%22type%22:%22loki%22,%22uid%22:%22loki%22%7D,%22editorMode%22:%22builder%22%7D%5D,%22range%22:%7B%22from%22:%221761810612000%22,%22to%22:%221761821412000%22%7D%7D%7D&schemaVersion=1&orgId=1

注入故障network bandwidth 类型解释:
Image

Image

Expected Behavior

No response

Steps to Reproduce

chaos network bandwidth test

Additional information

No response

Metadata

Metadata

Assignees

Labels

kind/bugSomething isn't workingseverity/s0Extreme impact: Cause the application to break down and seriously affect the use

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions