Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Console output and log message not accurate when import data with dup records using lightning tidb backend. #54081

Open
fubinzh opened this issue Jun 18, 2024 · 2 comments
Labels
component/lightning This issue is related to Lightning of TiDB. severity/minor type/bug The issue is confirmed as a bug.

Comments

@fubinzh
Copy link

fubinzh commented Jun 18, 2024

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

  1. Lightning import some data with few duplicate rows using tidb end, conflict.strategy = 'replace';
# cat /tmp/tidb-lightning.toml
[lightning]                                       
level = 'debug'                                   
[mydumper]                                        
filter = ['coms.PK_*']                            
[conflict]                                        
strategy = 'replace'     

2, Lighting import finishes

2. What did you expect to see? (Required)

Lightning import should succeed as there are only 5 duplicate rows.
And the info in lightning console output and log should be accurate.

3. What did you see instead (Required)

Lightning logs says that "Detect 5 conflict errors in total, please refer to table lightning_task_info.conflict_view for more details"
However, the conflict rows will not be recorded for strategy = "replace" in TiDB backend per https://docs.pingcap.com/zh/tidb/stable/tidb-lightning-configuration, also validated that there is no records in the tables actually.

[2024/06/17 15:04:36.751 +00:00] [INFO] [import.go:578] ["the whole procedure completed"] [takeTime=1h6m44.025684115s] []
[2024/06/17 15:04:36.751 +00:00] [WARN] [errormanager.go:1089] ["Detect 5 conflict errors in total, please refer to table `lightning_task_info`.`conflict_view` for more details"]
[2024/06/17 15:04:36.751 +00:00] [INFO] [main.go:110] ["tidb lightning exit"] [finished=true]

image
image

4. What is your TiDB version? (Required)

Release Version: v8.2.0-alpha
Git Commit Hash: 26d1096
Git Branch: heads/refs/tags/v8.2.0-alpha
Go Version: go1.21.10
UTC Build Time: 2024-06-17 11:41:27
Race Enabled: false

@fubinzh fubinzh added the type/bug The issue is confirmed as a bug. label Jun 18, 2024
@fubinzh
Copy link
Author

fubinzh commented Jun 18, 2024

/component lightning

@ti-chi-bot ti-chi-bot bot added the component/lightning This issue is related to Lightning of TiDB. label Jun 18, 2024
@fubinzh
Copy link
Author

fubinzh commented Jun 18, 2024

/severity minor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/lightning This issue is related to Lightning of TiDB. severity/minor type/bug The issue is confirmed as a bug.
Projects
None yet
Development

No branches or pull requests

1 participant