Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cdc panic: runtime error: invalid memory address or nil pointer dereference #10799

Closed
fubinzh opened this issue Mar 18, 2024 · 1 comment · Fixed by #10865
Closed

cdc panic: runtime error: invalid memory address or nil pointer dereference #10799

fubinzh opened this issue Mar 18, 2024 · 1 comment · Fixed by #10865
Assignees
Labels
affects-7.5 affects-8.0 area/ticdc Issues or PRs related to TiCDC. severity/major type/bug The issue is confirmed as a bug.

Comments

@fubinzh
Copy link

fubinzh commented Mar 18, 2024

What did you do?

This panic is seen after #10798

What did you expect to see?

cdc no panic

What did you see instead?

{"level":"warn","ts":"2024-03-17T03:44:29.64617Z","logger":"etcd-client","caller":"v3@v3.5.12/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0038b4000/upstream-pd-0.upstream-pd-peer.cdc-airbnb-simple-tian-gcp-tps-7500530-1-808.svc:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
{"level":"warn","ts":"2024-03-17T03:44:35.647313Z","logger":"etcd-client","caller":"v3@v3.5.12/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0038b4000/upstream-pd-0.upstream-pd-peer.cdc-airbnb-simple-tian-gcp-tps-7500530-1-808.svc:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
{"level":"warn","ts":"2024-03-17T03:44:41.648666Z","logger":"etcd-client","caller":"v3@v3.5.12/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0038b4000/upstream-pd-0.upstream-pd-peer.cdc-airbnb-simple-tian-gcp-tps-7500530-1-808.svc:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
{"level":"warn","ts":"2024-03-17T03:46:41.939417Z","logger":"etcd-client","caller":"v3@v3.5.12/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc003b921c0/upstream-pd-0.upstream-pd-peer.cdc-airbnb-simple-tian-gcp-tps-7500530-1-808.svc:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x3878de9]

goroutine 9470229 [running]:
github.com/pingcap/tiflow/cdc/kv.(*requestedStream).receive(0xc0706b43c0, {0x5c10998, 0xc0706b52c0}, 0xc0747ea580, 0xc06e0f2980, 0x1959945?, 0xc0706b48c0?)
        github.com/pingcap/tiflow/cdc/kv/shared_stream.go:225 +0x69
github.com/pingcap/tiflow/cdc/kv.(*requestedStream).run.func3()
        github.com/pingcap/tiflow/cdc/kv/shared_stream.go:187 +0x2e
golang.org/x/sync/errgroup.(*Group).Go.func1()
        golang.org/x/sync@v0.6.0/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 9470087
        golang.org/x/sync@v0.6.0/errgroup/errgroup.go:75 +0x96

Versions of the cluster

cdc version:
cdc version:
Release Version: v8.0.0
Git Commit Hash: 130403f
Git Branch: heads/refs/tags/v8.0.0
UTC Build Time: 2024-03-15 13:58:58
Go Version: go version go1.21.6 linux/amd64
Failpoint Build: false

@3AceShowHand
Copy link
Contributor

The client is closed and set to nil in the goroutine a, and read by the goroutine b, the cause the panic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment