Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

libp2p.Host leaks goroutines #2514

Closed
piersy opened this issue Aug 23, 2023 · 7 comments · Fixed by #2118
Closed

libp2p.Host leaks goroutines #2514

piersy opened this issue Aug 23, 2023 · 7 comments · Fixed by #2118
Labels
kind/bug A bug in existing code (including security flaws) P1 High: Likely tackled by core team if no one steps up status/blocked Unable to be worked further until needs are met

Comments

@piersy
Copy link
Contributor

piersy commented Aug 23, 2023 •
edited
Loading

Creating a new host and closing it leaks goroutines.

The following simple test produces the following errors when run: (goleak is a library that detects leaked goroutines VerifyNone fails the test if more goroutines exist at the end of the test than at the beginning)

package shutdown

import (
	"testing"

	"github.com/libp2p/go-libp2p"
	"github.com/stretchr/testify/require"
	"go.uber.org/goleak"
)

func TestShutdown(t *testing.T) {
	defer goleak.VerifyNone(t)
	h1, err := libp2p.New()
	require.NoError(t, err)
	defer h1.Close()
}
--- FAIL: TestShutdown (0.46s)
    leaks.go:78: found unexpected goroutines:
        [Goroutine 8 in state IO wait, with internal/poll.runtime_pollWait on top of the stack:
        goroutine 8 [IO wait]:
        internal/poll.runtime_pollWait(0x7f2328390a78, 0x72)
        	/home/pierspowlesland/sdk/go1.20/src/runtime/netpoll.go:306 +0x89
        internal/poll.(*pollDesc).wait(0xc000347c00?, 0xc?, 0x0)
        	/home/pierspowlesland/sdk/go1.20/src/internal/poll/fd_poll_runtime.go:84 +0x32
        internal/poll.(*pollDesc).waitRead(...)
        	/home/pierspowlesland/sdk/go1.20/src/internal/poll/fd_poll_runtime.go:89
        internal/poll.(*FD).RawRead(0xc000347c00, 0xc000392040)
        	/home/pierspowlesland/sdk/go1.20/src/internal/poll/fd_unix.go:766 +0x145
        net.(*rawConn).Read(0xc000205a20, 0x1c6c600?)
        	/home/pierspowlesland/sdk/go1.20/src/net/rawconn.go:43 +0x45
        golang.org/x/net/internal/socket.(*syscaller).recvmmsg(0xc0003a40d8, {0x1337e80?, 0xc000205a20?}, {0xc0003a6200?, 0x11fdb70?, 0x0?}, 0x40f6de?)
        	/home/pierspowlesland/go/pkg/mod/golang.org/x/net@v0.12.0/internal/socket/mmsghdr_unix.go:121 +0x94
        golang.org/x/net/internal/socket.(*Conn).recvMsgs(0xc000423be0, {0xc00047cdc0?, 0x8, 0x8}, 0x4?)
        	/home/pierspowlesland/go/pkg/mod/golang.org/x/net@v0.12.0/internal/socket/rawconn_mmsg.go:25 +0x185
        golang.org/x/net/internal/socket.(*Conn).RecvMsgs(...)
        	/home/pierspowlesland/go/pkg/mod/golang.org/x/net@v0.12.0/internal/socket/socket.go:267
        golang.org/x/net/ipv4.(*payloadHandler).ReadBatch(0xc000409a00, {0xc00047cdc0?, 0x5ac?, 0x5ac?}, 0xc00039cdc8?)
        	/home/pierspowlesland/go/pkg/mod/golang.org/x/net@v0.12.0/ipv4/batch.go:80 +0x57
        github.com/quic-go/quic-go.(*oobConn).ReadPacket(0xc0004fa000)
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/sys_conn_oob.go:164 +0x5ba
        github.com/quic-go/quic-go.(*Transport).listen(0xc0001da360, {0x7f23282249c8, 0xc0004fa000})
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/transport.go:299 +0x174
        created by github.com/quic-go/quic-go.(*Transport).init.func1
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/transport.go:211 +0x415
        
         Goroutine 63 in state select, with github.com/libp2p/go-libp2p/p2p/transport/quicreuse.(*reuse).gc on top of the stack:
        goroutine 63 [select]:
        github.com/libp2p/go-libp2p/p2p/transport/quicreuse.(*reuse).gc(0xc000408b40)
        	/home/pierspowlesland/projects/go-libp2p/p2p/transport/quicreuse/reuse.go:166 +0x11e
        created by github.com/libp2p/go-libp2p/p2p/transport/quicreuse.newReuse
        	/home/pierspowlesland/projects/go-libp2p/p2p/transport/quicreuse/reuse.go:141 +0x170
        
         Goroutine 64 in state select, with github.com/libp2p/go-libp2p/p2p/transport/quicreuse.(*reuse).gc on top of the stack:
        goroutine 64 [select]:
        github.com/libp2p/go-libp2p/p2p/transport/quicreuse.(*reuse).gc(0xc000408b90)
        	/home/pierspowlesland/projects/go-libp2p/p2p/transport/quicreuse/reuse.go:166 +0x11e
        created by github.com/libp2p/go-libp2p/p2p/transport/quicreuse.newReuse
        	/home/pierspowlesland/projects/go-libp2p/p2p/transport/quicreuse/reuse.go:141 +0x170
        
         Goroutine 69 in state IO wait, with internal/poll.runtime_pollWait on top of the stack:
        goroutine 69 [IO wait]:
        internal/poll.runtime_pollWait(0x7f2328390e38, 0x72)
        	/home/pierspowlesland/sdk/go1.20/src/runtime/netpoll.go:306 +0x89
        internal/poll.(*pollDesc).wait(0xc000347700?, 0xc00039aa01?, 0x0)
        	/home/pierspowlesland/sdk/go1.20/src/internal/poll/fd_poll_runtime.go:84 +0x32
        internal/poll.(*pollDesc).waitRead(...)
        	/home/pierspowlesland/sdk/go1.20/src/internal/poll/fd_poll_runtime.go:89
        internal/poll.(*FD).RawRead(0xc000347700, 0xc000392010)
        	/home/pierspowlesland/sdk/go1.20/src/internal/poll/fd_unix.go:766 +0x145
        net.(*rawConn).Read(0xc000205960, 0x7f23281f29d8?)
        	/home/pierspowlesland/sdk/go1.20/src/net/rawconn.go:43 +0x45
        golang.org/x/net/internal/socket.(*syscaller).recvmmsg(0xc0003a4048, {0x1337e80?, 0xc000205960?}, {0xc0003a6000?, 0x11fdb70?, 0x0?}, 0x40f6de?)
        	/home/pierspowlesland/go/pkg/mod/golang.org/x/net@v0.12.0/internal/socket/mmsghdr_unix.go:121 +0x94
        golang.org/x/net/internal/socket.(*Conn).recvMsgs(0xc000423360, {0xc00047c000?, 0x8, 0x8}, 0xe9e840?)
        	/home/pierspowlesland/go/pkg/mod/golang.org/x/net@v0.12.0/internal/socket/rawconn_mmsg.go:25 +0x185
        golang.org/x/net/internal/socket.(*Conn).RecvMsgs(...)
        	/home/pierspowlesland/go/pkg/mod/golang.org/x/net@v0.12.0/internal/socket/socket.go:267
        golang.org/x/net/ipv4.(*payloadHandler).ReadBatch(0xc000409230, {0xc00047c000?, 0x0?, 0x3?}, 0x0?)
        	/home/pierspowlesland/go/pkg/mod/golang.org/x/net@v0.12.0/ipv4/batch.go:80 +0x57
        github.com/quic-go/quic-go.(*oobConn).ReadPacket(0xc000227c20)
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/sys_conn_oob.go:164 +0x5ba
        github.com/quic-go/quic-go.(*Transport).listen(0xc00035a360, {0x7f23282249c8, 0xc000227c20})
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/transport.go:299 +0x174
        created by github.com/quic-go/quic-go.(*Transport).init.func1
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/transport.go:211 +0x415
        
         Goroutine 70 in state select, with github.com/quic-go/quic-go.(*Transport).runSendQueue on top of the stack:
        goroutine 70 [select]:
        github.com/quic-go/quic-go.(*Transport).runSendQueue(0xc00035a360)
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/transport.go:228 +0x113
        created by github.com/quic-go/quic-go.(*Transport).init.func1
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/transport.go:212 +0x456
        
         Goroutine 77 in state IO wait, with internal/poll.runtime_pollWait on top of the stack:
        goroutine 77 [IO wait]:
        internal/poll.runtime_pollWait(0x7f2328390d48, 0x72)
        	/home/pierspowlesland/sdk/go1.20/src/runtime/netpoll.go:306 +0x89
        internal/poll.(*pollDesc).wait(0xc000347980?, 0xc00038ba01?, 0x0)
        	/home/pierspowlesland/sdk/go1.20/src/internal/poll/fd_poll_runtime.go:84 +0x32
        internal/poll.(*pollDesc).waitRead(...)
        	/home/pierspowlesland/sdk/go1.20/src/internal/poll/fd_poll_runtime.go:89
        internal/poll.(*FD).RawRead(0xc000347980, 0xc000120010)
        	/home/pierspowlesland/sdk/go1.20/src/internal/poll/fd_unix.go:766 +0x145
        net.(*rawConn).Read(0xc0002059a0, 0x7f2328bce548?)
        	/home/pierspowlesland/sdk/go1.20/src/net/rawconn.go:43 +0x45
        golang.org/x/net/internal/socket.(*syscaller).recvmmsg(0xc00012c048, {0x1337e80?, 0xc0002059a0?}, {0xc00012e000?, 0x11fdb70?, 0x0?}, 0x40f6de?)
        	/home/pierspowlesland/go/pkg/mod/golang.org/x/net@v0.12.0/internal/socket/mmsghdr_unix.go:121 +0x94
        golang.org/x/net/internal/socket.(*Conn).recvMsgs(0xc0004239e0, {0xc00047c840?, 0x8, 0x8}, 0xe9e840?)
        	/home/pierspowlesland/go/pkg/mod/golang.org/x/net@v0.12.0/internal/socket/rawconn_mmsg.go:25 +0x185
        golang.org/x/net/internal/socket.(*Conn).RecvMsgs(...)
        	/home/pierspowlesland/go/pkg/mod/golang.org/x/net@v0.12.0/internal/socket/socket.go:267
        golang.org/x/net/ipv4.(*payloadHandler).ReadBatch(0xc000409500, {0xc00047c840?, 0x0?, 0x1?}, 0x0?)
        	/home/pierspowlesland/go/pkg/mod/golang.org/x/net@v0.12.0/ipv4/batch.go:80 +0x57
        github.com/quic-go/quic-go.(*oobConn).ReadPacket(0xc000227dd0)
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/sys_conn_oob.go:164 +0x5ba
        github.com/quic-go/quic-go.(*Transport).listen(0xc00035a7e0, {0x7f23282249c8, 0xc000227dd0})
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/transport.go:299 +0x174
        created by github.com/quic-go/quic-go.(*Transport).init.func1
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/transport.go:211 +0x415
        
         Goroutine 78 in state select, with github.com/quic-go/quic-go.(*Transport).runSendQueue on top of the stack:
        goroutine 78 [select]:
        github.com/quic-go/quic-go.(*Transport).runSendQueue(0xc00035a7e0)
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/transport.go:228 +0x113
        created by github.com/quic-go/quic-go.(*Transport).init.func1
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/transport.go:212 +0x456
        
         Goroutine 86 in state IO wait, with internal/poll.runtime_pollWait on top of the stack:
        goroutine 86 [IO wait]:
        internal/poll.runtime_pollWait(0x7f2328390b68, 0x72)
        	/home/pierspowlesland/sdk/go1.20/src/runtime/netpoll.go:306 +0x89
        internal/poll.(*pollDesc).wait(0xc000347a80?, 0xc000505a01?, 0x0)
        	/home/pierspowlesland/sdk/go1.20/src/internal/poll/fd_poll_runtime.go:84 +0x32
        internal/poll.(*pollDesc).waitRead(...)
        	/home/pierspowlesland/sdk/go1.20/src/internal/poll/fd_poll_runtime.go:89
        internal/poll.(*FD).RawRead(0xc000347a80, 0xc00007cb10)
        	/home/pierspowlesland/sdk/go1.20/src/internal/poll/fd_unix.go:766 +0x145
        net.(*rawConn).Read(0xc0002059e0, 0x0?)
        	/home/pierspowlesland/sdk/go1.20/src/net/rawconn.go:43 +0x45
        golang.org/x/net/internal/socket.(*syscaller).recvmmsg(0xc0000f2438, {0x1337e80?, 0xc0002059e0?}, {0xc000600000?, 0x11fdb70?, 0x0?}, 0x7f23283afe58?)
        	/home/pierspowlesland/go/pkg/mod/golang.org/x/net@v0.12.0/internal/socket/mmsghdr_unix.go:121 +0x94
        golang.org/x/net/internal/socket.(*Conn).recvMsgs(0xc000423ae0, {0xc00047cb00?, 0x8, 0x8}, 0xe9e840?)
        	/home/pierspowlesland/go/pkg/mod/golang.org/x/net@v0.12.0/internal/socket/rawconn_mmsg.go:25 +0x185
        golang.org/x/net/internal/socket.(*Conn).RecvMsgs(...)
        	/home/pierspowlesland/go/pkg/mod/golang.org/x/net@v0.12.0/internal/socket/socket.go:267
        golang.org/x/net/ipv4.(*payloadHandler).ReadBatch(0xc000409820, {0xc00047cb00?, 0x0?, 0x0?}, 0x0?)
        	/home/pierspowlesland/go/pkg/mod/golang.org/x/net@v0.12.0/ipv4/batch.go:80 +0x57
        github.com/quic-go/quic-go.(*oobConn).ReadPacket(0xc000227e60)
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/sys_conn_oob.go:164 +0x5ba
        github.com/quic-go/quic-go.(*Transport).listen(0xc00035aa20, {0x7f23282249c8, 0xc000227e60})
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/transport.go:299 +0x174
        created by github.com/quic-go/quic-go.(*Transport).init.func1
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/transport.go:211 +0x415
        
         Goroutine 87 in state select, with github.com/quic-go/quic-go.(*Transport).runSendQueue on top of the stack:
        goroutine 87 [select]:
        github.com/quic-go/quic-go.(*Transport).runSendQueue(0xc00035aa20)
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/transport.go:228 +0x113
        created by github.com/quic-go/quic-go.(*Transport).init.func1
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/transport.go:212 +0x456
        
         Goroutine 9 in state select, with github.com/quic-go/quic-go.(*Transport).runSendQueue on top of the stack:
        goroutine 9 [select]:
        github.com/quic-go/quic-go.(*Transport).runSendQueue(0xc0001da360)
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/transport.go:228 +0x113
        created by github.com/quic-go/quic-go.(*Transport).init.func1
        	/home/pierspowlesland/go/pkg/mod/github.com/quic-go/quic-go@v0.36.4/transport.go:212 +0x456
        ]
FAIL
FAIL	github.com/libp2p/go-libp2p/p2p/test/shutdown	0.464s
FAIL

This problem appears in: v0.29.2, v0.30.0 (and possibly prior versions, not yet tested)

Version Information 
github.com/libp2p/go-libp2p
cloud.google.com/go v0.65.0
cloud.google.com/go/bigquery v1.8.0
cloud.google.com/go/datastore v1.1.0
cloud.google.com/go/pubsub v1.3.1
cloud.google.com/go/storage v1.10.0
dmitri.shuralyov.com/app/changes v0.0.0-20180602232624-0a106ad413e3
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9
dmitri.shuralyov.com/html/belt v0.0.0-20180602232347-f7d459c86be0
dmitri.shuralyov.com/service/change v0.0.0-20181023043359-a85b471d5412
dmitri.shuralyov.com/state v0.0.0-20180228185332-28bcc343414c
git.apache.org/thrift.git v0.0.0-20180902110319-2566ecd5d999
github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96
github.com/BurntSushi/toml v0.3.1
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802
github.com/OneOfOne/xxhash v1.2.2
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751
github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d
github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239
github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6
github.com/benbjohnson/clock v1.3.5
github.com/beorn7/perks v1.0.1
github.com/bradfitz/go-smtpd v0.0.0-20170404230938-deb6d6237625
github.com/buger/jsonparser v0.0.0-20181115193947-bf1c66bbce23
github.com/census-instrumentation/opencensus-proto v0.2.1
github.com/cespare/xxhash v1.1.0
github.com/cespare/xxhash/v2 v2.2.0
github.com/chzyer/logex v1.1.10
github.com/chzyer/readline v1.5.1
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1
github.com/cilium/ebpf v0.9.1
github.com/client9/misspell v0.3.4
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f
github.com/containerd/cgroups v1.1.0
github.com/coreos/etcd v3.3.10+incompatible
github.com/coreos/go-etcd v2.0.0+incompatible
github.com/coreos/go-semver v0.2.0
github.com/coreos/go-systemd v0.0.0-20181012123002-c6f51f82210d
github.com/coreos/go-systemd/v22 v22.5.0
github.com/cpuguy83/go-md2man v1.0.10
github.com/cpuguy83/go-md2man/v2 v2.0.0
github.com/davecgh/go-spew v1.1.1
github.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c
github.com/decred/dcrd/crypto/blake256 v1.0.1
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0
github.com/dgraph-io/badger v1.6.2
github.com/dgraph-io/ristretto v0.0.2
github.com/dgryski/go-farm v0.0.0-20190423205320-6a90982ecee2
github.com/docker/go-units v0.5.0
github.com/dustin/go-humanize v1.0.0
github.com/elastic/gosigar v0.14.2
github.com/envoyproxy/go-control-plane v0.9.4
github.com/envoyproxy/protoc-gen-validate v0.1.0
github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568
github.com/flynn/noise v1.0.0
github.com/francoispqt/gojay v1.2.13
github.com/fsnotify/fsnotify v1.5.4
github.com/ghodss/yaml v1.0.0
github.com/gliderlabs/ssh v0.1.1
github.com/go-errors/errors v1.0.1
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4
github.com/go-kit/kit v0.9.0
github.com/go-kit/log v0.2.0
github.com/go-logfmt/logfmt v0.5.1
github.com/go-logr/logr v1.2.4
github.com/go-stack/stack v1.8.0
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572
github.com/godbus/dbus/v5 v5.1.0
github.com/gogo/protobuf v1.3.2
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e
github.com/golang/lint v0.0.0-20180702182130-06c8688daad7
github.com/golang/mock v1.6.0
github.com/golang/protobuf v1.5.3
github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db
github.com/google/btree v1.0.0
github.com/google/go-cmp v0.5.9
github.com/google/go-github v17.0.0+incompatible
github.com/google/go-querystring v1.0.0
github.com/google/gofuzz v1.0.0
github.com/google/gopacket v1.1.19
github.com/google/martian v2.1.0+incompatible
github.com/google/martian/v3 v3.0.0
github.com/google/pprof v0.0.0-20230705174524-200ffdc848b8
github.com/google/renameio v0.1.0
github.com/google/uuid v1.3.0
github.com/googleapis/gax-go v2.0.0+incompatible
github.com/googleapis/gax-go/v2 v2.0.5
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1
github.com/gorilla/websocket v1.5.0
github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7
github.com/grpc-ecosystem/grpc-gateway v1.5.0
github.com/hashicorp/golang-lru v0.5.1
github.com/hashicorp/golang-lru/v2 v2.0.2
github.com/hashicorp/hcl v1.0.0
github.com/hpcloud/tail v1.0.0
github.com/huin/goupnp v1.2.0
github.com/ianlancetaylor/demangle v0.0.0-20230524184225-eabc099b10ab
github.com/inconshreveable/mousetrap v1.0.0
github.com/ipfs/go-cid v0.4.1
github.com/ipfs/go-datastore v0.6.0
github.com/ipfs/go-detect-race v0.0.1
github.com/ipfs/go-ds-badger v0.3.0
github.com/ipfs/go-ds-leveldb v0.5.0
github.com/ipfs/go-ipfs-delay v0.0.0-20181109222059-70721b86a9a8
github.com/ipfs/go-ipfs-util v0.0.2
github.com/ipfs/go-log/v2 v2.5.1
github.com/jackpal/go-nat-pmp v1.0.2
github.com/jbenet/go-cienv v0.1.0
github.com/jbenet/go-temp-err-catcher v0.1.0
github.com/jbenet/goprocess v0.1.4
github.com/jellevandenhooff/dkim v0.0.0-20150330215556-f50fe3d243e1
github.com/jpillora/backoff v1.0.0
github.com/json-iterator/go v1.1.12
github.com/jstemmer/go-junit-report v0.9.1
github.com/julienschmidt/httprouter v1.3.0
github.com/kisielk/errcheck v1.5.0
github.com/kisielk/gotool v1.0.0
github.com/klauspost/compress v1.16.7
github.com/klauspost/cpuid/v2 v2.2.5
github.com/konsorten/go-windows-terminal-sequences v1.0.3
github.com/koron/go-ssdp v0.0.4
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515
github.com/kr/pretty v0.2.1
github.com/kr/pty v1.1.3
github.com/kr/text v0.1.0
github.com/libp2p/go-buffer-pool v0.1.0
github.com/libp2p/go-cidranger v1.1.0
github.com/libp2p/go-flow-metrics v0.1.0
github.com/libp2p/go-libp2p-asn-util v0.3.0
github.com/libp2p/go-libp2p-testing v0.12.0
github.com/libp2p/go-mplex v0.7.0
github.com/libp2p/go-msgio v0.3.0
github.com/libp2p/go-nat v0.2.0
github.com/libp2p/go-netroute v0.2.1
github.com/libp2p/go-openssl v0.1.0
github.com/libp2p/go-reuseport v0.3.0
github.com/libp2p/go-yamux/v4 v4.0.1
github.com/libp2p/zeroconf/v2 v2.2.0
github.com/lunixbochs/vtclean v1.0.0
github.com/magiconair/properties v1.8.0
github.com/mailru/easyjson v0.0.0-20190312143242-1de009706dbe
github.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd
github.com/mattn/go-isatty v0.0.19
github.com/mattn/go-pointer v0.0.1
github.com/matttproud/golang_protobuf_extensions v1.0.4
github.com/microcosm-cc/bluemonday v1.0.1
github.com/miekg/dns v1.1.55
github.com/mikioh/tcp v0.0.0-20190314235350-803a9b46060c
github.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b
github.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc
github.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1
github.com/minio/sha256-simd v1.0.1
github.com/mitchellh/go-homedir v1.1.0
github.com/mitchellh/mapstructure v1.1.2
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
github.com/modern-go/reflect2 v1.0.2
github.com/mr-tron/base58 v1.2.0
github.com/multiformats/go-base32 v0.1.0
github.com/multiformats/go-base36 v0.2.0
github.com/multiformats/go-multiaddr v0.10.1
github.com/multiformats/go-multiaddr-dns v0.3.1
github.com/multiformats/go-multiaddr-fmt v0.1.0
github.com/multiformats/go-multibase v0.2.0
github.com/multiformats/go-multicodec v0.9.0
github.com/multiformats/go-multihash v0.2.3
github.com/multiformats/go-multistream v0.4.1
github.com/multiformats/go-varint v0.0.7
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f
github.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86
github.com/neelance/sourcemap v0.0.0-20151028013722-8c68805598ab
github.com/nxadm/tail v1.4.8
github.com/onsi/ginkgo v1.16.5
github.com/onsi/ginkgo/v2 v2.11.0
github.com/onsi/gomega v1.27.8
github.com/opencontainers/runtime-spec v1.0.2
github.com/openzipkin/zipkin-go v0.1.1
github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58
github.com/pelletier/go-toml v1.2.0
github.com/pkg/errors v0.9.1
github.com/pmezard/go-difflib v1.0.0
github.com/prometheus/client_golang v1.14.0
github.com/prometheus/client_model v0.4.0
github.com/prometheus/common v0.37.0
github.com/prometheus/procfs v0.8.0
github.com/quic-go/qpack v0.4.0
github.com/quic-go/qtls-go1-18 v0.2.0
github.com/quic-go/qtls-go1-19 v0.3.3
github.com/quic-go/qtls-go1-20 v0.2.3
github.com/quic-go/quic-go v0.36.4
github.com/quic-go/webtransport-go v0.5.3
github.com/raulk/go-watchdog v1.3.0
github.com/rogpeppe/go-internal v1.3.0
github.com/russross/blackfriday v1.5.2
github.com/russross/blackfriday/v2 v2.0.1
github.com/sergi/go-diff v1.0.0
github.com/shurcooL/component v0.0.0-20170202220835-f88ec8f54cc4
github.com/shurcooL/events v0.0.0-20181021180414-410e4ca65f48
github.com/shurcooL/github_flavored_markdown v0.0.0-20181002035957-2122de532470
github.com/shurcooL/go v0.0.0-20180423040247-9e1955d9fb6e
github.com/shurcooL/go-goon v0.0.0-20170922171312-37c2f522c041
github.com/shurcooL/gofontwoff v0.0.0-20180329035133-29b52fc0a18d
github.com/shurcooL/gopherjslib v0.0.0-20160914041154-feb6d3990c2c
github.com/shurcooL/highlight_diff v0.0.0-20170515013008-09bb4053de1b
github.com/shurcooL/highlight_go v0.0.0-20181028180052-98c3abbbae20
github.com/shurcooL/home v0.0.0-20181020052607-80b7ffcb30f9
github.com/shurcooL/htmlg v0.0.0-20170918183704-d01228ac9e50
github.com/shurcooL/httperror v0.0.0-20170206035902-86b7830d14cc
github.com/shurcooL/httpfs v0.0.0-20171119174359-809beceb2371
github.com/shurcooL/httpgzip v0.0.0-20180522190206-b1c53ac65af9
github.com/shurcooL/issues v0.0.0-20181008053335-6292fdc1e191
github.com/shurcooL/issuesapp v0.0.0-20180602232740-048589ce2241
github.com/shurcooL/notifications v0.0.0-20181007000457-627ab5aea122
github.com/shurcooL/octicon v0.0.0-20181028054416-fa4f57f9efb2
github.com/shurcooL/reactions v0.0.0-20181006231557-f2e0b4ca5b82
github.com/shurcooL/sanitized_anchor_name v1.0.0
github.com/shurcooL/users v0.0.0-20180125191416-49c67e49c537
github.com/shurcooL/webdavfs v0.0.0-20170829043945-18c3829fa133
github.com/sirupsen/logrus v1.8.1
github.com/sourcegraph/annotate v0.0.0-20160123013949-f4cad6c6324d
github.com/sourcegraph/syntaxhighlight v0.0.0-20170531221838-bd320f5d308e
github.com/spacemonkeygo/spacelog v0.0.0-20180420211403-2296661a0572
github.com/spaolacci/murmur3 v1.1.0
github.com/spf13/afero v1.1.2
github.com/spf13/cast v1.3.0
github.com/spf13/cobra v0.0.5
github.com/spf13/jwalterweatherman v1.0.0
github.com/spf13/pflag v1.0.3
github.com/spf13/viper v1.3.2
github.com/stretchr/objx v0.5.0
github.com/stretchr/testify v1.8.4
github.com/syndtr/goleveldb v1.0.0
github.com/tarm/serial v0.0.0-20180830185346-98f6abe2eb07
github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8
github.com/urfave/cli v1.22.2
github.com/viant/assertly v0.4.8
github.com/viant/toolbox v0.24.0
github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77
github.com/yuin/goldmark v1.4.13
go.opencensus.io v0.22.4
go.uber.org/atomic v1.11.0
go.uber.org/dig v1.17.0
go.uber.org/fx v1.20.0
go.uber.org/goleak v1.1.12
go.uber.org/multierr v1.11.0
go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee
go.uber.org/zap v1.24.0
go4.org v0.0.0-20180809161055-417644f6feb5
golang.org/x/build v0.0.0-20190111050920-041ab4dc3f9d
golang.org/x/crypto v0.11.0
golang.org/x/exp v0.0.0-20230713183714-613f0c0eb8a1
golang.org/x/image v0.0.0-20190802002840-cff245a6509b
golang.org/x/lint v0.0.0-20200302205851-738671d3881b
golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028
golang.org/x/mod v0.12.0
golang.org/x/net v0.12.0
golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b
golang.org/x/perf v0.0.0-20180704124530-6e6d33e29852
golang.org/x/sync v0.3.0
golang.org/x/sys v0.10.0
golang.org/x/term v0.10.0
golang.org/x/text v0.11.0
golang.org/x/time v0.0.0-20191024005414-555d28b269f0
golang.org/x/tools v0.11.0
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1
google.golang.org/api v0.30.0
google.golang.org/appengine v1.6.6
google.golang.org/genproto v0.0.0-20200825200019-8632dd797987
google.golang.org/grpc v1.31.0
google.golang.org/protobuf v1.30.0
gopkg.in/alecthomas/kingpin.v2 v2.2.6
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c
gopkg.in/errgo.v2 v2.1.0
gopkg.in/fsnotify.v1 v1.4.7
gopkg.in/inf.v0 v0.9.1
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7
gopkg.in/yaml.v2 v2.4.0
gopkg.in/yaml.v3 v3.0.1
grpc.go4.org v0.0.0-20170609214715-11d0a25b4919
honnef.co/go/tools v0.0.1-2020.1.4
lukechampine.com/blake3 v1.2.1
rsc.io/binaryregexp v0.2.0
rsc.io/quote/v3 v3.1.0
rsc.io/sampler v1.3.0
sourcegraph.com/sourcegraph/go-diff v0.5.0
sourcegraph.com/sqs/pbtypes v0.0.0-20180604144634-d3ebe8f20ae4
@Jorropo
Copy link
Contributor

Jorropo commented Aug 23, 2023 •
edited
Loading

👍 I was able to repro this.
Adding a long sleep does not make this go away so it appears to be a real leak, not a lack of (*sync.WaitGroup).Wait.

func TestShutdown(t *testing.T) {
	defer goleak.VerifyNone(t)
	h1, err := libp2p.New()
	require.NoError(t, err)
	defer h1.Close()
	time.Sleep(time.Second * 10) // still fails
}

@piersy
Copy link
Contributor Author

piersy commented Aug 23, 2023

@Jorropo I tried the long sleep too, but no luck :(

@p-shahi
Copy link
Member

p-shahi commented Aug 23, 2023

Does this problem appear on v0.30.0 too?

@Jorropo
Copy link
Contributor

Jorropo commented Aug 23, 2023

Yes, I have tried it with v0.30.0.

@p-shahi
Copy link
Member

p-shahi commented Aug 23, 2023

@sukunrt mind taking a look?

@MarcoPolo
Copy link
Collaborator

I think it's related to this: #2118

We haven't done the work to stop some services that are started by FX. In this case it might be the quicreuse stuff.

@marten-seemann
Copy link
Contributor

Yes, using Fx would fix this. I’m not sure we’ll have the time to get this PR in before the v0.31 release, but if not, we should really prioritize it for the next one.

Adding a test using goleak seems like a good idea as well.

@p-shahi p-shahi mentioned this issue Aug 24, 2023
Closed
@marten-seemann marten-seemann added kind/bug A bug in existing code (including security flaws) status/blocked Unable to be worked further until needs are met P1 High: Likely tackled by core team if no one steps up labels Aug 25, 2023
@marten-seemann marten-seemann mentioned this issue Aug 28, 2023
Closed
25 tasks
@dindinw dindinw mentioned this issue Aug 29, 2023
Closed
5 tasks
@dennis-tra dennis-tra mentioned this issue Sep 19, 2023
Closed
@iand iand mentioned this issue Oct 2, 2023
Open
@dennis-tra dennis-tra mentioned this issue Oct 6, 2023
Merged
@marten-seemann marten-seemann mentioned this issue Oct 20, 2023
Merged
@sukunrt sukunrt mentioned this issue Oct 27, 2023
Closed
19 tasks
@MarcoPolo MarcoPolo linked a pull request Mar 21, 2024 that will close this issue
use Fx to start and stop the host, swarm, autorelay and quicreuse #2118
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug A bug in existing code (including security flaws) P1 High: Likely tackled by core team if no one steps up status/blocked Unable to be worked further until needs are met
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

use Fx to start and stop the host, swarm, autorelay and quicreuse libp2p/go-libp2p
5 participants
@MarcoPolo @marten-seemann @piersy @Jorropo @p-shahi