http2 failure in `UploadAsset`

[vchuravy]

We are using ghr to upload binary assets from Github to CI.

Currently I am seeing an increased failure rate:

Failed to upload one of assets: one of the goroutines failed: failed to upload asset: /tmp/jl_A641Uc/Clang_assert.v12.0.1.x86_64-linux-gnu-cxx11.tar.gz: failed to upload release asset: /tmp/jl_A641Uc/Clang_assert.v12.0.1.x86_64-linux-gnu-cxx11.tar.gz: Post https://uploads.github.com/repos/JuliaBinaryWrappers/Clang_assert_jll.jl/releases/49230189/assets?name=Clang_assert.v12.0.1.x86_64-linux-gnu-cxx11.tar.gz: http2: Transport: cannot retry err [stream error: stream ID 117; REFUSED_STREAM] after Request.Body was written; define Request.GetBody to avoid this error

AFAIK ghr uses UploadAsset: https://github.com/tcnksm/ghr/blob/93494744e6930d13e5c052eac9ca64a6027779a4/github.go#L184 from https://github.com/tcnksm/ghr/blob/93494744e6930d13e5c052eac9ca64a6027779a4/github.go#L209

and the error looks like golang/go#25009

[gmlewis]

Investigating... possibly related?
olivere/elastic#1443
https://sourcegraph.com/github.com/golang/net/-/commit/cffdcf672aee934982473246bc7e9a8ba446aa9b?visible=2

I remember we recently (<1year ago) made a couple changes with respect to the request:

• Clean up connection reuse code inside Do func as now Go HTTP packge does it internally #1645
• Read response body for TCP connection reuse #1576

but I'm not sure if these are related.

[vchuravy]

Turned out that is was hiding the actual errors:

Failed to upload one of assets: one of the goroutines failed: failed to upload asset: /tmp/jl_kxYk28/Clang_assert.v12.0.1.aarch64-apple-darwin.tar.gz: failed to upload release asset: /tmp/jl_kxYk28/Clang_assert.v12.0.1.aarch64-apple-darwin.tar.gz: Post https://uploads.github.com/repos/JuliaBinaryWrappers/Clang_assert_jll.jl/releases/49230189/assets?name=Clang_assert.v12.0.1.aarch64-apple-darwin.tar.gz: write tcp 128.30.92.237:33810->140.82.113.13:443: use of closed network connection
Failed to upload one of assets: one of the goroutines failed: failed to upload asset: /tmp/jl_kxYk28/Clang_assert.v12.0.1.x86_64-unknown-freebsd.tar.gz: failed to upload release asset: /tmp/jl_kxYk28/Clang_assert.v12.0.1.x86_64-unknown-freebsd.tar.gz: Post https://uploads.github.com/repos/JuliaBinaryWrappers/Clang_assert_jll.jl/releases/49230189/assets?name=Clang_assert.v12.0.1.x86_64-unknown-freebsd.tar.gz: write tcp 128.30.92.237:33988->140.82.113.13:443: use of closed network connection
Failed to upload one of assets: one of the goroutines failed: failed to upload asset: /tmp/jl_kxYk28/Clang_assert.v12.0.1.aarch64-linux-gnu-cxx11.tar.gz: failed to upload release asset: /tmp/jl_kxYk28/Clang_assert.v12.0.1.aarch64-linux-gnu-cxx11.tar.gz: Post https://uploads.github.com/repos/JuliaBinaryWrappers/Clang_assert_jll.jl/releases/49230189/assets?name=Clang_assert.v12.0.1.aarch64-linux-gnu-cxx11.tar.gz: write tcp 128.30.92.237:36032->140.82.114.14:443: write: broken pipe

[gmlewis]

Glad you found it. Closing issue.

[vchuravy]

I still think there is an issue here to be fixed. Upgrading to HTTP/2 shouldn't have swallowed these error messages.

[gmlewis]

Is anyone able to write a unit test that demonstrates the problem?

[wangyoucao577]

any update on this?

[caarlos0]

FWIW this happens occasionally for some goreleaser users as well (using v39).

goreleaser/goreleaser#2746

[cpanato]

having the same issue when using goreleaser

[blacktop]

Still getting this error