Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Goroutine leak when memory exceed quota #48007

Closed
wshwsh12 opened this issue Oct 26, 2023 · 1 comment · Fixed by #48133
Closed

Goroutine leak when memory exceed quota #48007

wshwsh12 opened this issue Oct 26, 2023 · 1 comment · Fixed by #48133
Labels
affects-5.3 This bug affects 5.3.x versions. affects-5.4 This bug affects 5.4.x versions. affects-6.1 affects-6.5 affects-7.1 affects-7.5 severity/major sig/execution SIG execution type/bug The issue is confirmed as a bug.

Comments

@wshwsh12
Copy link
Contributor

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

run command:

tiup playground v7.4.0 (or nightly)
tiup bench tpch prepare --db test --threads 5

run sql:

use test;
set @@tidb_mem_quota_query=128;
-- run tpch query11 many times
select ps_partkey, sum(ps_supplycost * ps_availqty) as value from partsupp, supplier, nation where ps_suppkey = s_suppkey and s_nationkey = n_nationkey and n_name = 'MOZAMBIQUE' group by ps_partkey having sum(ps_supplycost * ps_availqty) > ( select sum(ps_supplycost * ps_availqty) * 0.0001000000 from partsupp, supplier, nation where ps_suppkey = s_suppkey and s_nationkey = n_nationkey and n_name = 'MOZAMBIQUE' ) order by value desc;

2. What did you expect to see? (Required)

No goroutine leak.

3. What did you see instead (Required)

See http://127.0.0.1:10080/debug/pprof/goroutine?debug=2 and find some goroutines leak.

goroutine 156428 [select, 7 minutes]:
github.com/pingcap/tidb/store/copr.(*copIteratorWorker).sendToRespCh(0xc037115b20?, 0xc0373b73b0?, 0xc0164bba40?, 0xc0?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1003 +0xa6
github.com/pingcap/tidb/store/copr.(*copIteratorWorker).handleCopResponse(0xc037115b20, 0xc03ab0a378, 0xc0371a5320, 0xc0373b73b0, {0xc0360cb8c0, 0x156, 0x156}, 0xc030369a18?, 0xc03718d5e0, 0xc0164bba40, ...)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1442 +0x776
github.com/pingcap/tidb/store/copr.(*copIteratorWorker).handleCopPagingResult(0xc037115b20, 0x0?, 0x2?, 0xc0373b73b0, {0xc0360cb8c0?, 0xc0373889c0?, 0x0?}, 0x0?, 0xc03718d5e0, 0xc0164bba40, ...)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1342 +0x59
github.com/pingcap/tidb/store/copr.(*copIteratorWorker).handleTaskOnce(0xc037115b20, 0xc03ab0a378, 0xc03718d5e0, 0xc0164bba40?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1275 +0x10ec
github.com/pingcap/tidb/store/copr.(*copIteratorWorker).handleTask(0xc037115b20, {0x60b3de0, 0xc036c752f0}, 0xc00347ac58?, 0xc0164bba40)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1130 +0x18e
github.com/pingcap/tidb/store/copr.(*copIteratorWorker).run(0xc037115b20, {0x60b3de0, 0xc036c752f0})
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:817 +0xc5
created by github.com/pingcap/tidb/store/copr.(*copIterator).open in goroutine 150517
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:861 +0x9b

4. What is your TiDB version? (Required)

master, v7.4.0

@wshwsh12 wshwsh12 added the type/bug The issue is confirmed as a bug. label Oct 26, 2023
@ti-chi-bot ti-chi-bot bot added may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.1 may-affects-6.5 may-affects-7.1 labels Oct 27, 2023
@wshwsh12 wshwsh12 added may-affects-6.1 affects-5.3 This bug affects 5.3.x versions. affects-5.4 This bug affects 5.4.x versions. affects-6.1 and removed affects-6.1 labels Oct 31, 2023
@ti-chi-bot ti-chi-bot bot removed may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.1 labels Oct 31, 2023
@zanmato1984
Copy link
Contributor

This is very corner case, so adjusting the severity to major.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-5.3 This bug affects 5.3.x versions. affects-5.4 This bug affects 5.4.x versions. affects-6.1 affects-6.5 affects-7.1 affects-7.5 severity/major sig/execution SIG execution type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants