x/exp/cmd/gorelease: gets stuck in "Run go mod tidy" loop with cyclic module dependencies

[myitcv]

What version of Go are you using (go version)?

$ go version
go version devel +8869086d8f Thu Feb 4 04:46:49 2021 +0000 linux/amd64
$ go list -m golang.org/x/exp
golang.org/x/exp@v0.0.0-20210201131500-d352d2db2ceb

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/myitcv/.cache/go-build"
GOENV="/home/myitcv/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/myitcv/gostuff/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/myitcv/gostuff"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/myitcv/gos"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/myitcv/gos/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="devel +8869086d8f Thu Feb 4 04:46:49 2021 +0000"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1399875509=/tmp/go-build -gno-record-gcc-switches"

What did you do?

If you will excuse the large reproducer:

cd $(mktemp -d)
(
set -eu
git clone https://github.com/cuelang/cue
cd cue
git checkout fdccfbc52e248257d2ea20de1b2e0ed8589c3c8a
cat <<'EOD' > tools.go
// +build tools

package tools

import (
        _ "github.com/cue-sh/tools/cmd/cueckoo"
        _ "golang.org/x/exp/cmd/gorelease"
        _ "golang.org/x/tools/cmd/goimports"
)
EOD
go get golang.org/x/exp@v0.0.0-20210201131500-d352d2db2ceb
go list -m golang.org/x/exp
go get github.com/cue-sh/tools/cmd/cueckoo@v0.0.0-20210205194434-b3293e86064a
go mod tidy
go run golang.org/x/exp/cmd/gorelease
)

What did you expect to see?

Success.

What did you see instead?

go.mod: the following requirements are needed
        cuelang.org/go@v0.3.0-beta.3
Run 'go mod tidy' to add missing requirements.

cc @jayconrod

[t0mpl]

Hi everyone,

Just updated gopls and same here, my projects are red with the message

err: exit status 1: stderr: go: updates to go.sum needed, disabled by -mod=readonly
: packages.Load error
go.sum is out of sync with go.mod. Please update it or run "go mod tidy".

no matter if i run go mod tidy or delete go.sum and go install, nothing helps

Can't help because i have no idea what happens here, good luck guys

[jayconrod]

@ThomasP1988 Your errors don't seem related to this bug (not related to gorelease). Please open a new issue or ask in #gopls or #modules on Gopher Slack.

[jayconrod]

This an interesting one. It looks like at this commit, cue transitively depends on cuelang.org/cue@v0.3.0-beta.3. The base version is v0.2.2 (inferred as @latest), and the release version should be v0.3.0 (bumping minor version due to API changes).

When gorelease loads packages, it creates a module zip file as the go command would and extracts it to a temporary directory. It then creates a go.mod file in a separate temporary directory like the one below.

module gorelease-load-module

go 1.16

require cuelang.org/go v0.0.0-gorelease

replace cuelang.org/go v0.0.0-gorelease => $tmpDirWhereZipWasExtracted

// other requirements copied from go.mod

This lets the go command (via packages.Load) load packages as if it were in a dependent module, ignoring replace and exclude directives.

We're running into a problem here with that fake v0.0.0-gorelease version. After MVS, v0.3.0-beta.3 is selected instead, and gorelease complains that the requirements are wrong.

I think gorelease should use a fake version that's higher than all other versions. At the moment, v0.3.0-beta.4 is the highest version, so it could be something like v0.3.1-gorelease.

cc @jadekler

[jeanbza]

I investigated this today. I was able to repro it with the (excellent!) repro steps above:

$ go run golang.org/x/exp/cmd/gorelease
[...]
Inferred base version: v0.4.0
go.mod: the following requirements are needed
	cuelang.org/go@v0.3.0-beta.3
Run 'go mod tidy' to add missing requirements.
exit status 1

But, when I ran it on the latest version of gorelease, the problem seems to go away:

$ go get -u golang.org/x/exp/cmd/gorelease
go: downloading golang.org/x/exp v0.0.0-20210604131234-3d6432c72ee9
go: downloading golang.org/x/tools v0.1.2
go: downloading golang.org/x/sys v0.0.0-20210603125802-9665404d3644
go get: upgraded github.com/google/go-cmp v0.4.0 => v0.5.5
go get: upgraded github.com/stretchr/testify v1.4.0 => v1.7.0
go get: upgraded golang.org/x/exp v0.0.0-20210201131500-d352d2db2ceb => v0.0.0-20210604131234-3d6432c72ee9
go get: upgraded golang.org/x/net v0.0.0-20200301022130-244492dfa37a => v0.0.0-20210405180319-a5a99cb37ef4
go get: upgraded golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527 => v0.0.0-20210603125802-9665404d3644
go get: upgraded golang.org/x/text v0.3.2 => v0.3.3
go get: upgraded golang.org/x/tools v0.0.0-20200612220849-54c614fe050c => v0.1.2
go get: upgraded golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 => v0.0.0-20200804184101-5ec99f83aff1
go get: upgraded gopkg.in/yaml.v3 v3.0.0-20200121175148-a6ecf24a6d71 => v3.0.0-20200313102051-9f266ea9e77c
$ git co .
$ go run golang.org/x/exp/cmd/gorelease
[...]
Inferred base version: v0.4.0
Suggested version: v0.5.0
deklerk at deklerk1 in /tmp/tmp.S1me6UfJjG/cue on (HEAD detached at fdccfbc5)
$ echo $?
0

I suspect this has inadvertently been fixed in one of the other recent changes we've made.

---

@myitcv could you confirm that you no longer see this issue?

[myitcv]

@myitcv could you confirm that you no longer see this issue?

Looks good to me, thanks! I'll close on that basis, but please reopen if I grabbed the wrong end of the stick there.

[jeanbza]

Thanks Paul! :)