e2e tests broken #619
Description
I've ran make test-e2e with gpu deployment and seems like the e2e tests are broken.
more specifically, the curl command from the curl pod to envoy svc is not working.
I assume something in the names/configuration has changed recently, but further investigation is needed in order to see what is the exact problem.
Nirs-MBP-2:gateway-api-inference-extension nirro$ make test-e2e
MANIFEST_PATH=/Users/nirro/dev/github.com/fork/gateway-api-inference-extension/config/manifests/vllm/gpu-deployment.yaml go test ./test/e2e/epp/ -v -ginkgo.v
=== RUN TestAPIs
Running Suite: End To End Test Suite - /Users/nirro/dev/github.com/fork/gateway-api-inference-extension/test/e2e/epp
====================================================================================================================
Random Seed: 1743444796
Will run 1 of 1 specs
------------------------------
[BeforeSuite]
/Users/nirro/dev/github.com/fork/gateway-api-inference-extension/test/e2e/epp/e2e_suite_test.go:101
STEP: Setting up the test suite @ 03/31/25 21:13:16.457
STEP: Creating test infrastructure @ 03/31/25 21:13:16.459
STEP: Ensuring MANIFEST_PATH environment variable is set @ 03/31/25 21:13:16.459
STEP: Creating CRD resource from manifest: ../../../config/crd/bases/inference.networking.x-k8s.io_inferencepools.yaml @ 03/31/25 21:13:16.459
STEP: Reading YAML file: ../../../config/crd/bases/inference.networking.x-k8s.io_inferencepools.yaml @ 03/31/25 21:13:16.46
STEP: Decoded GVK: apiextensions.k8s.io/v1, Kind=CustomResourceDefinition @ 03/31/25 21:13:16.461
STEP: Checking CRD inferencepools.inference.networking.x-k8s.io status is: Established @ 03/31/25 21:13:18.426
STEP: Creating CRD resource from manifest: ../../../config/crd/bases/inference.networking.x-k8s.io_inferencemodels.yaml @ 03/31/25 21:13:18.757
STEP: Reading YAML file: ../../../config/crd/bases/inference.networking.x-k8s.io_inferencemodels.yaml @ 03/31/25 21:13:18.758
STEP: Decoded GVK: apiextensions.k8s.io/v1, Kind=CustomResourceDefinition @ 03/31/25 21:13:18.76
STEP: Checking CRD inferencemodels.inference.networking.x-k8s.io status is: Established @ 03/31/25 21:13:19.186
STEP: Creating inference extension resources from manifest: ../../../config/manifests/inferencepool-resources.yaml @ 03/31/25 21:13:19.362
STEP: Reading YAML file: ../../../config/manifests/inferencepool-resources.yaml @ 03/31/25 21:13:19.363
STEP: Decoded GVK: inference.networking.x-k8s.io/v1alpha2, Kind=InferencePool @ 03/31/25 21:13:19.364
STEP: Decoded GVK: /v1, Kind=Service @ 03/31/25 21:13:21.691
STEP: Decoded GVK: apps/v1, Kind=Deployment @ 03/31/25 21:13:21.898
STEP: Decoded GVK: rbac.authorization.k8s.io/v1, Kind=ClusterRole @ 03/31/25 21:13:22.082
STEP: Decoded GVK: rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding @ 03/31/25 21:13:22.243
STEP: Checking if deployment default/vllm-llama3-8b-instruct-epp status is: Available @ 03/31/25 21:13:22.899
STEP: Creating client resources from manifest: ../../testdata/client.yaml @ 03/31/25 21:13:32.853
STEP: Reading YAML file: ../../testdata/client.yaml @ 03/31/25 21:13:32.853
STEP: Decoded GVK: /v1, Kind=Pod @ 03/31/25 21:13:32.855
STEP: Checking pod default/curl status is: Ready @ 03/31/25 21:13:33.199
STEP: Creating envoy proxy resources from manifest: ../../testdata/envoy.yaml @ 03/31/25 21:13:34.218
STEP: Reading YAML file: ../../testdata/envoy.yaml @ 03/31/25 21:13:34.218
STEP: Decoded GVK: /v1, Kind=ConfigMap @ 03/31/25 21:13:34.22
STEP: Decoded GVK: apps/v1, Kind=Deployment @ 03/31/25 21:13:34.537
STEP: Decoded GVK: /v1, Kind=Service @ 03/31/25 21:13:34.719
STEP: Checking if deployment default/envoy status is: Available @ 03/31/25 21:13:35.248
STEP: Ensuring the model server manifest points to an existing file @ 03/31/25 21:13:37.694
STEP: Reading YAML file: /Users/nirro/dev/github.com/fork/gateway-api-inference-extension/config/manifests/vllm/gpu-deployment.yaml @ 03/31/25 21:13:37.694
STEP: Ensuring the HF_TOKEN environment variable is set @ 03/31/25 21:13:37.696
STEP: Reading YAML file: ../../testdata/model-secret.yaml @ 03/31/25 21:13:37.696
STEP: Replacing placeholder secret data with HF_TOKEN environment variable @ 03/31/25 21:13:37.696
STEP: Creating model server secret resource @ 03/31/25 21:13:37.696
STEP: Decoded GVK: /v1, Kind=Secret @ 03/31/25 21:13:37.696
STEP: Creating model server resources from manifest: /Users/nirro/dev/github.com/fork/gateway-api-inference-extension/config/manifests/vllm/gpu-deployment.yaml @ 03/31/25 21:13:38.02
STEP: Decoded GVK: apps/v1, Kind=Deployment @ 03/31/25 21:13:38.021
STEP: Decoded GVK: /v1, Kind=ConfigMap @ 03/31/25 21:13:38.2
STEP: Checking if deployment default/vllm-llama3-8b-instruct status is: Available @ 03/31/25 21:13:38.532
[BeforeSuite] PASSED [222.192 seconds]
------------------------------
InferencePool when The Inference Extension is running Should route traffic to target model servers
/Users/nirro/dev/github.com/fork/gateway-api-inference-extension/test/e2e/epp/e2e_test.go:45
STEP: Waiting for the namespace to exist. @ 03/31/25 21:16:58.648
STEP: Ensuring namespace exists: default @ 03/31/25 21:16:58.648
STEP: Creating an InferenceModel resource @ 03/31/25 21:16:58.808
STEP: Ensuring the InferenceModel resource exists in the namespace @ 03/31/25 21:16:58.975
STEP: Verifying connectivity through the inference extension @ 03/31/25 21:16:59.139
panic: test timed out after 10m0s
running tests:
TestAPIs (10m0s)
goroutine 479 [running]:
testing.(*M).startAlarm.func1()
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/testing/testing.go:2484 +0x308
created by time.goFunc
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/time/sleep.go:215 +0x38
goroutine 1 [chan receive, 10 minutes]:
testing.(*T).Run(0x14000103c00, {0x10145046c?, 0x1400011fb38?}, 0x101c07950)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/testing/testing.go:1859 +0x388
testing.runTests.func1(0x14000103c00)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/testing/testing.go:2279 +0x40
testing.tRunner(0x14000103c00, 0x1400011fc68)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/testing/testing.go:1792 +0xe4
testing.runTests(0x140001446a8, {0x102c77f50, 0x1, 0x1}, {0x140002de0e0?, 0xf?, 0x102cbeb80?})
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/testing/testing.go:2277 +0x3ec
testing.(*M).Run(0x140004d14a0)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/testing/testing.go:2142 +0x588
main.main()
_testmain.go:45 +0x90
goroutine 5 [select, 6 minutes]:
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode(_, {0x7, 0x4, {0x10148833c, 0x2c}, 0x14000051100, {{0x1021c3ec3, 0x59}, 0x2d, {0x0, ...}, ...}, ...}, ...)
/Users/nirro/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.23.3/internal/suite.go:916 +0xc58
github.com/onsi/ginkgo/v2/internal.(*group).attemptSpec(0x140006a5800, 0x1, {{0x14000282008?, 0x140003078f0?, 0x2?}, 0x0?})
/Users/nirro/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.23.3/internal/group.go:199 +0x8bc
github.com/onsi/ginkgo/v2/internal.(*group).run(0x140006a5800, {0x1400047ce20?, 0x0?, 0x0?})
/Users/nirro/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.23.3/internal/group.go:349 +0x804
github.com/onsi/ginkgo/v2/internal.(*Suite).runSpecs(0x14000130a88, {0x101463067, 0x15}, {0x102ce7a60, 0x0, 0x0}, {0x1400004e004, 0x4d}, 0x0, {0x140002de320, ...})
/Users/nirro/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.23.3/internal/suite.go:489 +0x7c0
github.com/onsi/ginkgo/v2/internal.(*Suite).Run(_, {_, _}, {_, _, _}, {_, _}, _, {_, ...}, ...)
/Users/nirro/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.23.3/internal/suite.go:130 +0x2e8
github.com/onsi/ginkgo/v2.RunSpecs({0x101c11a60, 0x14000103dc0}, {0x101463067, 0x15}, {0x0, 0x0, 0x0})
/Users/nirro/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.23.3/core_dsl.go:300 +0x858
sigs.k8s.io/gateway-api-inference-extension/test/e2e/epp.TestAPIs(0x14000103dc0)
/Users/nirro/dev/github.com/fork/gateway-api-inference-extension/test/e2e/epp/e2e_suite_test.go:96 +0x50
testing.tRunner(0x14000103dc0, 0x101c07950)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/testing/testing.go:1792 +0xe4
created by testing.(*T).Run in goroutine 1
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/testing/testing.go:1851 +0x374
goroutine 22 [syscall, 10 minutes]:
os/signal.signal_recv()
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/runtime/sigqueue.go:149 +0x2c
os/signal.loop()
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/os/signal/signal_unix.go:23 +0x1c
created by os/signal.Notify.func1.1 in goroutine 5
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/os/signal/signal.go:152 +0x28
goroutine 7 [select, 10 minutes]:
github.com/onsi/ginkgo/v2/internal/interrupt_handler.(*InterruptHandler).registerForInterrupts.func2(0x0?)
/Users/nirro/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.23.3/internal/interrupt_handler/interrupt_handler.go:131 +0x70
created by github.com/onsi/ginkgo/v2/internal/interrupt_handler.(*InterruptHandler).registerForInterrupts in goroutine 5
/Users/nirro/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.23.3/internal/interrupt_handler/interrupt_handler.go:128 +0x170
goroutine 8 [select, 10 minutes]:
github.com/onsi/ginkgo/v2/internal.RegisterForProgressSignal.func1()
/Users/nirro/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.23.3/internal/progress_report.go:32 +0x8c
created by github.com/onsi/ginkgo/v2/internal.RegisterForProgressSignal in goroutine 5
/Users/nirro/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.23.3/internal/progress_report.go:30 +0xd8
goroutine 75 [select]:
k8s.io/apimachinery/pkg/util/httpstream/spdy.(*connection).sendPings(0x140007ae540, 0x0?)
/Users/nirro/go/pkg/mod/k8s.io/apimachinery@v0.32.3/pkg/util/httpstream/spdy/connection.go:192 +0x98
created by k8s.io/apimachinery/pkg/util/httpstream/spdy.newConnection in goroutine 514
/Users/nirro/go/pkg/mod/k8s.io/apimachinery@v0.32.3/pkg/util/httpstream/spdy/connection.go:96 +0x180
goroutine 76 [sync.WaitGroup.Wait, 6 minutes]:
sync.runtime_SemacquireWaitGroup(0x14000681f01?)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/runtime/sema.go:110 +0x2c
sync.(*WaitGroup).Wait(0x1400062de30)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/sync/waitgroup.go:118 +0x70
k8s.io/client-go/tools/remotecommand.(*streamProtocolV4).stream(0x14000472180, {0x149be8d60?, 0x140007ae540?})
/Users/nirro/go/pkg/mod/k8s.io/client-go@v0.32.3/tools/remotecommand/v4.go:72 +0xd4
k8s.io/client-go/tools/remotecommand.(*spdyStreamExecutor).StreamWithContext.func1()
/Users/nirro/go/pkg/mod/k8s.io/client-go@v0.32.3/tools/remotecommand/spdy.go:160 +0x88
created by k8s.io/client-go/tools/remotecommand.(*spdyStreamExecutor).StreamWithContext in goroutine 514
/Users/nirro/go/pkg/mod/k8s.io/client-go@v0.32.3/tools/remotecommand/spdy.go:154 +0x170
goroutine 514 [select, 6 minutes]:
k8s.io/client-go/tools/remotecommand.(*spdyStreamExecutor).StreamWithContext(0x14000420d80?, {0x101c34e68, 0x102ce7a60}, {{0x0, 0x0}, {0x101c11de0, 0x14000427710}, {0x101c11de0, 0x140004277d0}, 0x0, ...})
/Users/nirro/go/pkg/mod/k8s.io/client-go@v0.32.3/tools/remotecommand/spdy.go:163 +0x1d4
sigs.k8s.io/gateway-api-inference-extension/test/utils.ExecCommandInPod({0x101c34e68, 0x102ce7a60}, 0x14000129688, 0x14000481c70, 0x140006028c0, {0x10144ec4e, 0x7}, {0x10144bd8a, 0x4}, {0x10144bd8a, ...}, ...)
/Users/nirro/dev/github.com/fork/gateway-api-inference-extension/test/utils/utils.go:267 +0x2d0
sigs.k8s.io/gateway-api-inference-extension/test/e2e/epp.init.func3.3.1.2()
/Users/nirro/dev/github.com/fork/gateway-api-inference-extension/test/e2e/epp/e2e_test.go:65 +0x98
reflect.Value.call({0x101803a60?, 0x140008875c0?, 0x14000500008?}, {0x10144bea6, 0x4}, {0x102ce7a60, 0x0, 0x101a24d00?})
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/reflect/value.go:581 +0x978
reflect.Value.Call({0x101803a60?, 0x140008875c0?, 0x14000055bf8?}, {0x102ce7a60?, 0x101803a60?, 0x140008875c0?})
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/reflect/value.go:365 +0x94
github.com/onsi/gomega/internal.(*AsyncAssertion).buildActualPoller.func3()
/Users/nirro/go/pkg/mod/github.com/onsi/gomega@v1.36.3/internal/async_assertion.go:325 +0xd4
github.com/onsi/gomega/internal.(*AsyncAssertion).match(0x140001433b0, {0x101c21b68, 0x102ce7a60}, 0x1, {0x0, 0x0, 0x0})
/Users/nirro/go/pkg/mod/github.com/onsi/gomega@v1.36.3/internal/async_assertion.go:398 +0x104
github.com/onsi/gomega/internal.(*AsyncAssertion).Should(0x140001433b0, {0x101c21b68, 0x102ce7a60}, {0x0, 0x0, 0x0})
/Users/nirro/go/pkg/mod/github.com/onsi/gomega@v1.36.3/internal/async_assertion.go:145 +0x7c
sigs.k8s.io/gateway-api-inference-extension/test/e2e/epp.init.func3.3.1()
/Users/nirro/dev/github.com/fork/gateway-api-inference-extension/test/e2e/epp/e2e_test.go:87 +0x484
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x140008a3c80?, 0x0?})
/Users/nirro/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.23.3/internal/node.go:475 +0x24
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/Users/nirro/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.23.3/internal/suite.go:894 +0x7c
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 5
/Users/nirro/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.23.3/internal/suite.go:881 +0xa8c
goroutine 74 [IO wait]:
internal/poll.runtime_pollWait(0x149da8438, 0x72)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/runtime/netpoll.go:351 +0xa0
internal/poll.(*pollDesc).wait(0x140006e1b80?, 0x14000292a80?, 0x0)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/internal/poll/fd_poll_runtime.go:84 +0x28
internal/poll.(*pollDesc).waitRead(...)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x140006e1b80, {0x14000292a80, 0xa80, 0xa80})
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/internal/poll/fd_unix.go:165 +0x1fc
net.(*netFD).Read(0x140006e1b80, {0x14000292a80?, 0x1400073e8a8?, 0x10007eccc?})
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/net/fd_posix.go:55 +0x28
net.(*conn).Read(0x14000472148, {0x14000292a80?, 0x300292a80?, 0x149b92118?})
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/net/net.go:194 +0x34
crypto/tls.(*atLeastReader).Read(0x1400042a240, {0x14000292a80?, 0x1400073e968?, 0x100358a14?})
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/crypto/tls/conn.go:809 +0x40
bytes.(*Buffer).ReadFrom(0x14000080638, {0x101c15340, 0x1400042a240})
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/bytes/buffer.go:211 +0x90
crypto/tls.(*Conn).readFromUntil(0x14000080388, {0x149d6f318, 0x14000472148}, 0x1400073ea10?)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/crypto/tls/conn.go:831 +0xcc
crypto/tls.(*Conn).readRecordOrCCS(0x14000080388, 0x0)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/crypto/tls/conn.go:629 +0x35c
crypto/tls.(*Conn).readRecord(...)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/crypto/tls/conn.go:591
crypto/tls.(*Conn).Read(0x14000080388, {0x14000802584, 0x4, 0x1001ca92c?})
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/crypto/tls/conn.go:1385 +0x15c
io.ReadAtLeast({0x149d70560, 0x14000080388}, {0x14000802584, 0x4, 0x4}, 0x4)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/io/io.go:335 +0xa0
io.ReadFull(...)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/io/io.go:354
encoding/binary.Read({0x149d70560, 0x14000080388}, {0x101c3a6e0, 0x102ce7a60}, {0x1017a6ea0, 0x1400073ee24})
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/encoding/binary/binary.go:262 +0x7c
github.com/moby/spdystream/spdy.(*Framer).ReadFrame(0x14000319aa0)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/spdy/read.go:164 +0x4c
github.com/moby/spdystream.(*idleAwareFramer).ReadFrame(0x1400051a780)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:167 +0x24
github.com/moby/spdystream.(*Connection).Serve(0x140005ca580, 0x14000051870)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:346 +0x204
created by k8s.io/apimachinery/pkg/util/httpstream/spdy.newConnection in goroutine 514
/Users/nirro/go/pkg/mod/k8s.io/apimachinery@v0.32.3/pkg/util/httpstream/spdy/connection.go:94 +0x124
goroutine 73 [select]:
github.com/moby/spdystream.(*idleAwareFramer).monitor(0x1400051a780)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:77 +0xa0
created by github.com/moby/spdystream.NewConnection in goroutine 514
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:269 +0x338
goroutine 530 [chan receive, 6 minutes]:
github.com/moby/spdystream.(*Connection).Serve.func1(0x14000032a20)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:327 +0x2c
created by github.com/moby/spdystream.(*Connection).Serve in goroutine 74
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:326 +0x124
goroutine 531 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0x1400094e990, 0x188)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/runtime/sema.go:597 +0x150
sync.(*Cond).Wait(0x1400094e980)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/sync/cond.go:71 +0xc4
github.com/moby/spdystream.(*PriorityFrameQueue).Pop(0x14000032a20)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/priority.go:102 +0x80
github.com/moby/spdystream.(*Connection).frameHandler(0x140005ca580, 0x14000032a20, 0x14000051870)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:419 +0x34
github.com/moby/spdystream.(*Connection).Serve.func2(0x14000456380?)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:336 +0x58
created by github.com/moby/spdystream.(*Connection).Serve in goroutine 74
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:332 +0x58
goroutine 532 [chan receive, 6 minutes]:
github.com/moby/spdystream.(*Connection).Serve.func1(0x14000032a50)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:327 +0x2c
created by github.com/moby/spdystream.(*Connection).Serve in goroutine 74
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:326 +0x124
goroutine 533 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0x1400094e9d0, 0x10)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/runtime/sema.go:597 +0x150
sync.(*Cond).Wait(0x1400094e9c0)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/sync/cond.go:71 +0xc4
github.com/moby/spdystream.(*PriorityFrameQueue).Pop(0x14000032a50)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/priority.go:102 +0x80
github.com/moby/spdystream.(*Connection).frameHandler(0x140005ca580, 0x14000032a50, 0x14000051870)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:419 +0x34
github.com/moby/spdystream.(*Connection).Serve.func2(0x0?)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:336 +0x58
created by github.com/moby/spdystream.(*Connection).Serve in goroutine 74
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:332 +0x58
goroutine 534 [chan receive, 6 minutes]:
github.com/moby/spdystream.(*Connection).Serve.func1(0x14000032a80)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:327 +0x2c
created by github.com/moby/spdystream.(*Connection).Serve in goroutine 74
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:326 +0x124
goroutine 535 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0x1400094ea10, 0xf)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/runtime/sema.go:597 +0x150
sync.(*Cond).Wait(0x1400094ea00)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/sync/cond.go:71 +0xc4
github.com/moby/spdystream.(*PriorityFrameQueue).Pop(0x14000032a80)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/priority.go:102 +0x80
github.com/moby/spdystream.(*Connection).frameHandler(0x140005ca580, 0x14000032a80, 0x14000051870)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:419 +0x34
github.com/moby/spdystream.(*Connection).Serve.func2(0x0?)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:336 +0x58
created by github.com/moby/spdystream.(*Connection).Serve in goroutine 74
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:332 +0x58
goroutine 536 [chan receive, 6 minutes]:
github.com/moby/spdystream.(*Connection).Serve.func1(0x14000032ab0)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:327 +0x2c
created by github.com/moby/spdystream.(*Connection).Serve in goroutine 74
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:326 +0x124
goroutine 537 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0x1400094ea50, 0x10)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/runtime/sema.go:597 +0x150
sync.(*Cond).Wait(0x1400094ea40)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/sync/cond.go:71 +0xc4
github.com/moby/spdystream.(*PriorityFrameQueue).Pop(0x14000032ab0)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/priority.go:102 +0x80
github.com/moby/spdystream.(*Connection).frameHandler(0x140005ca580, 0x14000032ab0, 0x14000051870)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:419 +0x34
github.com/moby/spdystream.(*Connection).Serve.func2(0x0?)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:336 +0x58
created by github.com/moby/spdystream.(*Connection).Serve in goroutine 74
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:332 +0x58
goroutine 538 [chan receive, 6 minutes]:
github.com/moby/spdystream.(*Connection).Serve.func1(0x14000032ae0)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:327 +0x2c
created by github.com/moby/spdystream.(*Connection).Serve in goroutine 74
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:326 +0x124
goroutine 539 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0x1400094ea90, 0xf)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/runtime/sema.go:597 +0x150
sync.(*Cond).Wait(0x1400094ea80)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/sync/cond.go:71 +0xc4
github.com/moby/spdystream.(*PriorityFrameQueue).Pop(0x14000032ae0)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/priority.go:102 +0x80
github.com/moby/spdystream.(*Connection).frameHandler(0x140005ca580, 0x14000032ae0, 0x14000051870)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:419 +0x34
github.com/moby/spdystream.(*Connection).Serve.func2(0x7361726f6c2d7570?)
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:336 +0x58
created by github.com/moby/spdystream.(*Connection).Serve in goroutine 74
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/connection.go:332 +0x58
goroutine 540 [select, 6 minutes]:
github.com/moby/spdystream.(*Stream).Read(0x140000b6c80, {0x14000915200, 0x200, 0x3a66222c7d7b3a22?})
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/stream.go:96 +0x78
io.ReadAll({0x149bd0b10, 0x140000b6c80})
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/io/io.go:712 +0x78
k8s.io/client-go/tools/remotecommand.watchErrorStream.func1()
/Users/nirro/go/pkg/mod/k8s.io/client-go@v0.32.3/tools/remotecommand/errorstream.go:41 +0x60
created by k8s.io/client-go/tools/remotecommand.watchErrorStream in goroutine 76
/Users/nirro/go/pkg/mod/k8s.io/client-go@v0.32.3/tools/remotecommand/errorstream.go:38 +0xac
goroutine 541 [select, 6 minutes]:
github.com/moby/spdystream.(*Stream).Read(0x140007e3720, {0x14000915400, 0x200, 0x10011a498?})
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/stream.go:96 +0x78
bytes.(*Buffer).ReadFrom(0x14000427710, {0x149bd0b10, 0x140007e3720})
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/bytes/buffer.go:211 +0x90
io.copyBuffer({0x101c11de0, 0x14000427710}, {0x149bd0b10, 0x140007e3720}, {0x0, 0x0, 0x0})
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/io/io.go:415 +0x14c
io.Copy(...)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/io/io.go:388
k8s.io/client-go/tools/remotecommand.(*streamProtocolV2).copyStdout.func1()
/Users/nirro/go/pkg/mod/k8s.io/client-go@v0.32.3/tools/remotecommand/v2.go:149 +0xcc
created by k8s.io/client-go/tools/remotecommand.(*streamProtocolV2).copyStdout in goroutine 76
/Users/nirro/go/pkg/mod/k8s.io/client-go@v0.32.3/tools/remotecommand/v2.go:141 +0x84
goroutine 542 [select]:
github.com/moby/spdystream.(*Stream).Read(0x140000b6d20, {0x1400001b4a6, 0xb5a, 0x10011a498?})
/Users/nirro/go/pkg/mod/github.com/moby/spdystream@v0.5.0/stream.go:96 +0x78
bytes.(*Buffer).ReadFrom(0x140004277d0, {0x149bd0b10, 0x140000b6d20})
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/bytes/buffer.go:211 +0x90
io.copyBuffer({0x101c11de0, 0x140004277d0}, {0x149bd0b10, 0x140000b6d20}, {0x0, 0x0, 0x0})
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/io/io.go:415 +0x14c
io.Copy(...)
/Users/nirro/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/io/io.go:388
k8s.io/client-go/tools/remotecommand.(*streamProtocolV2).copyStderr.func1()
/Users/nirro/go/pkg/mod/k8s.io/client-go@v0.32.3/tools/remotecommand/v2.go:166 +0xcc
created by k8s.io/client-go/tools/remotecommand.(*streamProtocolV2).copyStderr in goroutine 76
/Users/nirro/go/pkg/mod/k8s.io/client-go@v0.32.3/tools/remotecommand/v2.go:161 +0x8c
FAIL sigs.k8s.io/gateway-api-inference-extension/test/e2e/epp 600.719s
FAIL
make: *** [test-e2e] Error 1