Multipart reader regression in go 1.17

[Stebalien]

Checklist

• This is a bug report, not a question. Ask questions on discuss.ipfs.io.
• I have searched on the issue tracker for my bug.
• I am running the latest go-ipfs version or have an issue updating.

Installation method

built from source

Version

No response

Config

No response

Description

See the test failure in ipfs/go-ipfs-files#40:

--- FAIL: TestMultipartFiles (0.00s)
    helpers_test.go:122: [2] found end of directory, expected files.Event{kind:0, name:"nested", value:"some content"}
FAIL
FAIL	github.com/ipfs/go-ipfs-files	0.007s
FAIL

This isn't a flake so something is going wrong. I'm not sure if this affects production, but we should figure it out before releasing.

[BigLep]

Next step:

1. Confirm this only manifests with go 1.17 and if so release with 1.16.

[Stebalien]

Confirm this only manifests with go 1.17 and if so release with 1.16.

The CI passes with 1.17 but not with 1.16, so I believe we can consider this confirmed. But yeah, we can just not use 1.17 when we release.

[marten-seemann]

The CI passes with 1.17 but not with 1.16

I believe this should be the other way around: The CI passes with 1.16 but not with 1.17.

Is this a bug in the standard library?

[Stebalien]

Uh, yes, you're right. And it might be a bug, or they might be getting more strict.

[marten-seemann]

This is probably due to golang/go@784ef4c5313, which fixes golang/go#45789.

[marten-seemann]

Now the question is, have we been using the multipart reader in an incorrect way, potentially creating a security issue as described in the Go issue, or is this something we should work around by reimplementing the Go 1.16 behavior? @Stebalien?

[Stebalien]

We've been using it exactly as we intended.

[Stebalien]

Fix in ipfs/go-ipfs-files#42

[lidel]

Remaining work: tag a new release of https://github.com/ipfs/go-ipfs-files and switch go-ipfs to it in a PR that closes this issue + the one about timestamps.