cmd/go: allow replacement version to be omitted if the target module has a required version

[dmitris]

Currently you must add a version when entering a non-filesystem (remote) replacement for a module. A foolish attempt to put replace github.com/fsnotify/fsnotify => gitmirror.corp.xyz.com/fsnotify/fsnotify provokes a rebuke from go mod verify:
go.mod:42: replacement module without version must be directory path (rooted or starting with ./ or ../)

• so you have to fix it to be: replace github.com/fsnotify/fsnotify => gitmirror.corp.xyz.com/fsnotify/fsnotify v1.4.7.

However, there is already a version specification for that package in go.mod the require statement, for example:
require github.com/fsnotify/fsnotify v1.4.7
This seems to be the reasonable default value for the missing version in the replace directive for the same package - "if no replacement version is given, use the same as in the require directive for that specific package`.

What would be especially nice is when upgrading a package such as github.com/fsnotify/fsnotify to the future v1..4.8 version, one would not need to first run go get -u github.com/fsnotify/fsnotify and then have to look up the new version and manually update the old version to the new one in the replace section (or worse, forgetting to do it and ending up with the unintended replacement with the old version).

@thepudds said on Slack that he wanted to suggest this as well. @bcmills @rsc - does it seem reasonable to you?

[bcmills]

there is already a version specification for that package in go.mod [in a] require statement

I'm pretty sure that won't work at all right now: as a consequence of the fix for #26607, you cannot both require a module and use it as the target of a replace.

[bcmills]

This is certainly worth considering when we allow overlapping replacements, though. (That's #26904.)

[thepudds]

Just expanding on the use case briefly:

Currently it is an error to not have a right-hand version in a replace directive if the replacement is a remote repo.

If the right-hand version in a replace was allowed to be optional when the replacement is a remote repo, it could be natural then to tell the go tooling how to obtain a copy from a mirror, for example:

replace github.com/foo/bar => gitmirror.corp.com/foo/bar

And then have the actual versions used could be automatically be resolved "normally" via MVS and the normal version picking rules, but hitting gitmirror.corp.com rather than github.com to get the VCS metadata (and actual source code, etc.). In other words, the replacement would apply to the repo, not the version.

That would be less fragile than something like a go.mod file with 15 replace lines that effectively hard code the versions (with sample here taken from a real-world example):

replace (
   github.com/ardielle/ardielle-go => mirror.foobar.com/ardielle/ardielle-go.git v1.5.1
   github.com/aws/aws-sdk-go => mirror.foobar.com/aws/aws-sdk-go.git v1.15.49 // indirect
   github.com/coreos/go-oidc => mirror.foobar.com/coreos/go-oidc.git v2.0.0+incompatible
   github.com/golang/protobuf => mirror.foobar.com/golang/protobuf.git v1.2.0
   ...

I suspect that would help make modules a better upgrade for users that previously relied on glide repo directives or dep source and similar.

Perhaps the ultimate answer might end up being "use Athens", or "roll your own GOPROXY", but seems at least worth considering including this in the core go tool.

[thepudds]

@bcmills not the most urgent question, but could you expand slightly on what you meant here:

I'm pretty sure that won't work at all right now: as a consequence of the fix for #26607, you cannot both require a module and use it as the target of a replace.

In particular, what do you mean by a "target" there -- the left side module in the replace, or the right side module?

Going back to the first example from @dmitris above, I think this is what he was trying to say as a theoretical go.mod if this issue/suggestion here was implemented:

require github.com/fsnotify/fsnotify v1.4.7
replace github.com/fsnotify/fsnotify => gitmirror.corp.xyz.com/fsnotify/fsnotify

Which piece of that would violate the "forbid use of one module with two different paths" restriction from the fix for #26607?

(Of course, that is not a valid go.mod today because of there is no version on the right-hand side of the replace, but that's the topic of this issue/suggestion here).

[bcmills]

In particular, what do you mean by a "target" there -- the left side module in the replace, or the right side module?

The right side module.

Which piece of that would violate the "forbid use of one module with two different paths" restriction from the fix for #26607?

I'm not sure. I think I misunderstood the original report!

[dmitris]

@bcmills - to make sure we are on the same page, do you think some variation of the following change as clarified by @thepudds on Oct. 12 would be possible:

    require github.com/fsnotify/fsnotify v1.4.7
    replace github.com/fsnotify/fsnotify => gitmirror.corp.xyz.com/fsnotify/fsnotify

(instead of having to specify the replacement as currently: replace github.com/fsnotify/fsnotify => gitmirror.corp.xyz.com/fsnotify/fsnotify v1.4.7)?

It you try that now (with go1.12.1), you get an error: go.mod:29: replacement module without version must be directory path (rooted or starting with ./ or ../).

A coworker just asked me today about the best way to update "in one shot" both the require and replace sections of go.mod to a more recent version of some dependencies - the best I could think of was "use go get to update the require section, then a text editor to manually modify all the corresponding lines in the replace section." This is not only lots of manual labor but also very error-prone - a common error I see is forgetting to change some replace lines consistently and therefore inadvertently using the old version (and making go.mod a confused mess). Yes, one could write a linter for go.mod to flag the differences in versions between require and replace - and then hope everyone remembers to use it every time changing go.mod - but much better and easier would be to have the go tool helping here.

I understand that there is like a problem with replace github.com/fsnotify/fsnotify => gitmirror.corp.xyz.com/fsnotify/fsnotify syntax since go consideres gitmirror.corp.xyz.com/fsnotify/fsnotify the file path to the replacement module, therefore the error "must be directory path...".
Maybe we could add some "syntax sugar" like replace github.com/fsnotify/fsnotify => gitmirror.corp.xyz.com/fsnotify/fsnotify v* meaning use the same as version as in the corresponding require line (I believe "*" is never a valid version spec otherwise).

[directionless]

I have a use case that I think fits this, but might not? I often find myself forking open source modules, and wanting to build. The relative file path works fine for my local checkout, but fails horrible for CI or inside-docker builds.

I want to say something simple, like replace github.com/fsnotify/fsnotify => replace github.com/directionless/fsnotify@branchname but instead I'm not even sure what I do. Mostly I get frustrated