proposal: cmd/go: subcommands to add and remove modules from the module cache

[bcmills]

For a number of use-cases, it would be helpful to be able to upload modules to the module cache from source code (not just zip files!) in a local directory or repository.

Some examples:

• Testing changes involving cyclic module requirements (cmd/go: support simultaneous edits of interdependent modules #27542, cmd/go: ensure that 'go mod tidy' and go get -u do not introduce ambiguous imports #27899).
• Bootstrapping a server to serve the module containing itself (cmd/go: mod vendor should work offline #28029, but that can be worked around using replace directives).
• Storing and distributing private modules using version control systems that don't work well with zip files (related to cmd/go: add modvendor sub-command #27618).

To support those use-cases, I propose the following subcommands:

• go mod pack [MODULE[@VERSION]] DIR: construct a module in the module cache from the module source code rooted at DIR (at version VERSION). If the MODULE is omitted, it is inferred from DIR/go.mod. If @VERSION is provided, it must be a valid semantic version, and go mod pack fails if that version already exists with different contents. If @VERSION is omitted, DIR must be within a supported version control repository, and go mod pack will attempt to infer the version from the repo state (commits and tags).

• go mod unpack MODULE[@VERSION] DIR: download the contents of MODULE to DIR. If @VERSION is omitted, use the active version from the main module (if any), or latest if no version is active. (In contrast to go mod vendor, go mod unpack would unpack the entire contents of the module — not just the packages in the import graph of the main module.)

• go clean -m MODULE[@VERSION]: remove MODULE@VERSION from the module cache. If run within a module, also remove the corresponding entry from its go.sum file. If @VERSION is omitted, remove all versions of MODULE from the module cache (and go.sum file).

CC @hyangah @jadekler @rsc @myitcv @thepudds @rasky @rogpeppe @FiloSottile

[bcmills]

A way to explicitly populate the module cache from source might also help in cases where the original source path is blocked or unavailable but the code is available from a trusted mirror (as in #28652).

[rsc]

We have go mod download to add to the module cache.
We have go mod clean -modcache to clear it.
Do we really need more fine-grained control?
I fear that will make people manage it more.

[rsc]

Never mind I didn't understand the problem being solved.

[gopherbot]

Change https://golang.org/cl/153819 mentions this issue: cmd/go/internal/modfetch: skip symlinks in (*coderepo).Zip

[rsc]

Ping @bcmills to summarize our discussion from 2 weeks ago about alternatives to meet the need you are trying to address here.

[gopherbot]

Change https://golang.org/cl/153822 mentions this issue: cmd/go/internal/modfetch: skip symlinks in (*coderepo).Zip