cmd/go: go get ignores go.mod from remote repo, can escalate into a security vuln

[karalabe]

Happens with all Go versions ever supporting modules.

---

Repro: run outside of a Go module, I just want to install a Go binary:

go install -v github.com/ethereum/go-ethereum/cmd/geth

What happens:

go/src/github.com/ethereum/go-ethereum/metrics/influxdb/influxdb.go:10:2: cannot find package "github.com/influxdata/influxdb/client" in any of:
	/opt/google/go/src/github.com/influxdata/influxdb/client (from $GOROOT)
	/tmp/go/src/github.com/influxdata/influxdb/client (from $GOPATH)

Why? Because Go flat out ignores the module file distributed in that repo; and upstream nuked their repository to pieces so there's no code there any more.

---

Doing it manually works:

$ git clone https://github.com/ethereum/go-ethereum
$ cd go-ethereum
$ go install ./cmd/geth

---

This is apparently the same bug that was a release blocker in Go 1.12, and then just ignored and locked: #24250

[karalabe]

Here's a way to escalate this bug into a security vulnerability: https://github.com/karalabe/go-issue-38196-keygen

[seankhliao]

I believe this is go in gopath mode working as it always has been

if you want to install in module mode GO111MODULE=on go get github.com/karalabe/go-issue-38196-keygen

related #30515

[ALTree]

This is apparently the same bug that was a release blocker in Go 1.12, and then just ignored and locked: #24250

The issue you linked (#24250) was marked as a 1.12 release blocker, fixed by CL 148517, which was included in 1.12, and thus closed by the gopherbot when the commit was merged.

This is the opposite of "ignored and locked". Did you link the wrong issue?

[karalabe]

I believe this is go in gopath mode working as it always has been

if you want to install in module mode GO111MODULE=on go get github.com/karalabe/go-issue-38196-keygen

@seankhliao

$ GO111MODULE=on go get github.com/karalabe/go-issue-38196-keygen
go: downloading github.com/karalabe/go-issue-38196-keygen v0.0.0-20200401121913-5c894df7c1c5
go: github.com/karalabe/go-issue-38196-keygen upgrade => v0.0.0-20200401121913-5c894df7c1c5

$ go-issue-38196-keygen
Generating crypto key: insecure key

Also as far as I know as of Go 1.13, the default is on, so that should not make much of a difference.

The issue you linked (#24250) was marked as a 1.12 release blocker, fixed by CL 148517, which was included in 1.12, and thus closed by the gopherbot when the commit was merged.
This is the opposite of "ignored and locked". Did you link the wrong issue?

Hmm, I did not see that part, only the last lines in the issue:

@golang golang locked and limited conversation to collaborators on Feb 26
@gopherbot gopherbot added the FrozenDueToAge label on Feb 26

The last comments from that issue seem to be exactly this: installing a binary will not use the proper go modules.

[karalabe]

@sean-jc Hmm, scratch that, apparently go get without GOBIN stashed the binary somewhere weird:

$ GOBIN=`pwd` GO111MODULE=on go get github.com/karalabe/go-issue-38196-keygen
go: github.com/karalabe/go-issue-38196-keygen upgrade => v0.0.0-20200401121913-5c894df7c1c5

$ ./go-issue-38196-keygen 
Generating crypto key: secure key

[seankhliao]

Also as far as I know as of Go 1.13, the default is on, so that should not make much of a difference.

GO111MODULE continues to default to auto as of go1.14

[myitcv]

Please can I humbly make a suggestion:

Go flat out ignores

then just ignored and locked

Well, shit.

(last one quoted from the linked repro repository).

There is no need to resort to such hyperbole, not least when you might not be in possession of all the facts.

Instead, asking questions is more powerful way of conveying the same point:

Go appears to ignore... if I've understood this correctly?

appears to have been closed without a solution?

This is somewhat surprising

[karalabe]

@myitcv Fair points. It's just a bit frustrating when I see the following timelines:

rsc added NeedsFix release-blocker labels on Apr 9, 2019
rsc modified the milestones: Go1.13, Go1.14 on May 9, 2019
jayconrod modified the milestones: Go1.14, Go1.15 on Nov 11, 2019

It's fine to keep pushing a feature/fix if it doesn't make it into a release. It's just strange that something is important enough to be tagged a release-blocker, and then pushed 3 releases and counting. It seems to indicate that there is a lack of interest to solve the issue, even though everyone agrees on its severity. Hence why my strong language.

[myitcv]

Understood, but I think the first step should be asking questions to confirm you have the correct understanding/interpretation; that costs nothing and builds rapport with people, rather than potentially putting people's backs up.

Your frustration is shared by a number of people through #30515 and friends. This particular topic has been raised on the golang-tools calls many times. Unfortunately more pressing issues have pushed this down the priority list.