Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: remove leak option #7777

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

tests: remove leak option #7777

wants to merge 1 commit into from

Conversation

HuSharp
Copy link
Member

@HuSharp HuSharp commented Jan 30, 2024

What problem does this PR solve?

Issue Number: Close #7782

What is changed and how does it work?

for example, we will meet goroutine leak which top stack is runtime_pollWait which resulted from dashboard

Goroutine 1362 in state IO wait, with internal/poll.runtime_pollWait on top of the stack:
goroutine 1362 [IO wait]:
internal/poll.runtime_pollWait(0x14dc55908, 0x72)
	/opt/homebrew/opt/go/libexec/src/runtime/netpoll.go:343 +0xa0
internal/poll.(*pollDesc).wait(0x14008d8e580?, 0x0?, 0x0)
	/opt/homebrew/opt/go/libexec/src/internal/poll/fd_poll_runtime.go:84 +0x28
internal/poll.(*pollDesc).waitRead(...)
	/opt/homebrew/opt/go/libexec/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0x14008d8e580)
	/opt/homebrew/opt/go/libexec/src/internal/poll/fd_unix.go:611 +0x250
net.(*netFD).accept(0x14008d8e580)
	/opt/homebrew/opt/go/libexec/src/net/fd_unix.go:172 +0x28
net.(*TCPListener).accept(0x140088d1840)
	/opt/homebrew/opt/go/libexec/src/net/tcpsock_posix.go:152 +0x28
net.(*TCPListener).Accept(0x140088d1840)
	/opt/homebrew/opt/go/libexec/src/net/tcpsock.go:315 +0x2c
github.com/pingcap/tidb-dashboard/pkg/tidb.(*proxy).run(0x14008d9e1e0, {0x104f79778?, 0x14007fd5c20})
	/Users/pingcap/go/pkg/mod/github.com/pingcap/tidb-dashboard@v0.0.0-20240111062855-41f7c8011953/pkg/tidb/proxy.go:227 +0x37c
created by github.com/pingcap/tidb-dashboard/pkg/tidb.(*Forwarder).Start in goroutine 1296
	/Users/pingcap/go/pkg/mod/github.com/pingcap/tidb-dashboard@v0.0.0-20240111062855-41f7c8011953/pkg/tidb/forwarder.go:57 +0x1d4

besides we will meet goroutine leak error which top stack is runtime_pollWait as well, but the root case is go.etcd.io/etcd/pkg/transport.timeoutConn.Read which is different with dashboard

internal/poll.runtime_pollWait(0x10f8cc2a0, 0x72)
	/opt/homebrew/opt/go/libexec/src/runtime/netpoll.go:343 +0xa0
internal/poll.(*pollDesc).wait(0x140016ac400?, 0x1400159a000?, 0x0)
	/opt/homebrew/opt/go/libexec/src/internal/poll/fd_poll_runtime.go:84 +0x28
internal/poll.(*pollDesc).waitRead(...)
	/opt/homebrew/opt/go/libexec/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x140016ac400, {0x1400159a000, 0x1000, 0x1000})
	/opt/homebrew/opt/go/libexec/src/internal/poll/fd_unix.go:164 +0x200
net.(*netFD).Read(0x140016ac400, {0x1400159a000?, 0x14000436160?, 0x140013dcc60?})
	/opt/homebrew/opt/go/libexec/src/net/fd_posix.go:55 +0x28
net.(*conn).Read(0x140016b81c0, {0x1400159a000?, 0x14001141bd8?, 0x1?})
	/opt/homebrew/opt/go/libexec/src/net/net.go:179 +0x34
go.etcd.io/etcd/pkg/transport.timeoutConn.Read({{0x106cf7120?, 0x140016b81c0?}, 0x14001141c78?, 0x1041334cc?}, {0x1400159a000?, 0x104133074?, 0x1400039b068?})
	/Users/pingcap/go/pkg/mod/go.etcd.io/etcd@v0.5.0-alpha.5.0.20220915004622-85b640cee793/pkg/transport/timeout_conn.go:43 +0xa8
net/http.(*persistConn).Read(0x140013dcc60, {0x1400159a000?, 0x104133570?, 0x140013caf00?})
	/opt/homebrew/opt/go/libexec/src/net/http/transport.go:1954 +0x50
bufio.(*Reader).fill(0x14001073d40)
	/opt/homebrew/opt/go/libexec/src/bufio/bufio.go:113 +0xf8
bufio.(*Reader).Peek(0x14001073d40, 0x1)
	/opt/homebrew/opt/go/libexec/src/bufio/bufio.go:151 +0x60
net/http.(*persistConn).readLoop(0x140013dcc60)
	/opt/homebrew/opt/go/libexec/src/net/http/transport.go:2118 +0x14c
created by net/http.(*Transport).dialConn in goroutine 316
	/opt/homebrew/opt/go/libexec/src/net/http/transport.go:1776 +0x1144

When we use top stack to ignore, it results in dashboard errors not being exposed.
A better way to treat these two issues is to:

  • wait for the etcd timeout (because the goroutine is still exiting)
  • Troubleshooting the dashboard
    So we need a more fine-grained to check

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Code changes

Release note

None.

Copy link
Contributor

ti-chi-bot bot commented Jan 30, 2024

[REVIEW NOTIFICATION]

This pull request has not been approved.

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

Copy link
Contributor

ti-chi-bot bot commented Jan 30, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-linked-issue do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note-none Denotes a PR that doesn't merit a release note. labels Jan 30, 2024
@ti-chi-bot ti-chi-bot bot requested review from lhy1024 and nolouch January 30, 2024 06:19
@ti-chi-bot ti-chi-bot bot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Jan 30, 2024
@HuSharp HuSharp removed request for nolouch and lhy1024 January 30, 2024 06:19
@ti-chi-bot ti-chi-bot bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jan 30, 2024
@HuSharp HuSharp force-pushed the fix_leak branch 2 times, most recently from 2475d52 to 580a77d Compare January 30, 2024 15:00
@ti-chi-bot ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 30, 2024
@HuSharp HuSharp force-pushed the fix_leak branch 5 times, most recently from 51b2fc2 to 4581dd2 Compare January 31, 2024 10:17
@ti-chi-bot ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jan 31, 2024
@HuSharp HuSharp force-pushed the fix_leak branch 5 times, most recently from 30ddb17 to f87e9bb Compare January 31, 2024 12:40
Copy link

codecov bot commented Jan 31, 2024

Codecov Report

Merging #7777 (654845c) into master (f0699ba) will increase coverage by 0.14%.
Report is 71 commits behind head on master.
The diff coverage is 55.69%.

❗ Current head 654845c differs from pull request most recent head b97929a. Consider uploading reports for the commit b97929a to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7777      +/-   ##
==========================================
+ Coverage   73.45%   73.59%   +0.14%     
==========================================
  Files         432      433       +1     
  Lines       47843    47871      +28     
==========================================
+ Hits        35142    35230      +88     
+ Misses       9663     9620      -43     
+ Partials     3038     3021      -17     
Flag Coverage Δ
unittests 73.59% <55.69%> (+0.14%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@HuSharp HuSharp force-pushed the fix_leak branch 2 times, most recently from a8e2b09 to 3b90e9b Compare February 1, 2024 02:45
@HuSharp HuSharp force-pushed the fix_leak branch 3 times, most recently from fc9ab98 to 654845c Compare February 1, 2024 06:56
@ti-chi-bot ti-chi-bot bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 6, 2024
@ti-chi-bot ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Feb 6, 2024
Signed-off-by: husharp <jinhao.hu@pingcap.com>
@ti-chi-bot ti-chi-bot bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 29, 2024
Copy link
Contributor

ti-chi-bot bot commented Mar 29, 2024

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enhance go leak check
1 participant