cmd/go: add a workspace mode

[matloob]

Detailed Design Doc: Proposal: Multi-Module Workspaces in cmd/go

Demo Video: Go Workspaces Proposal Demo

High level overview:

I propose adding a new workspace mode in the go command for editing multiple modules. The presence of a go.work file in the working directory or a containing directory will put the go command into workspace mode. The go.work file specifies a set of local modules that comprise a workspace. When invoked in workspace mode, the go command will always select these modules and a consistent set of dependencies.

This is intended to help with workflows making changes across multiple modules and with editor support for those workflows.

This is what an example go.work file would look like:

go 1.17

directory (
    ./baz // foo.org/bar/baz
    ./tools // golang.org/x/tools
)

replace golang.org/x/net => example.com/fork/net v1.4.5

This adds two modules to the workspace. If the user's current working directory is is under the directory containing this go.work, the go command will be in workspace mode and use both the modules defined by ./baz/go.mod and ./tools/go.mod as main modules, regardless of which module the user is currently in (unless workspace mode is turned off or a different workspace is chosen with the proposed new -workfile flag). The replace would override any replaces
in the main modules' go.mod files.

Related issues

#32394 x/tools/gopls: support multi-module workspaces

Issue #32394 is about gopls' support for multi-module workspaces. gopls currently allows users to provide a "workspace root" which is a directory it searches for go.mod files to build a supermodule from. Alternatively, users can create a gopls.mod file in their workspace root that gopls will use as its supermodule. This proposal creates a concept of a workspace that is similar to that gopls that is understood by the go command so that users can have a consistent configuration across their editor and direct invocations of the go command.

#44347 proposal: cmd/go: support local experiments with interdependent modules; then retire GOPATH

Issue #44347 proposes adding a GOTINKER mode to the go command. Under the proposal, if GOTINKER is set to a directory, the go command will resolve import paths and dependencies in modules by looking first in a GOPATH-structured tree under the GOTINKER directory before looking at the module cache. This would allow users who want to have a GOPATH like workflow to build a GOPATH at GOTINKER, but still resolve most of their dependencies (those not in the GOTINKER tree) using the standard module resolution system. It also provides for a multi-module workflow for users who put their modules under GOTINKER and work in those modules.

This proposal also tries to provide some aspects of the GOPATH workflow and to help with multi-module workflows. A user could put the modules that they would put under GOTINKER in that proposal into their go.work files to get a similar experience to the one they'd get under the GOTINKER proposal. A major difference between the proposals is that in GOTINKER modules would be found by their paths under the GOTINKER tree instead of being explicitly listed in the go.work file. But both proposals provide for a set of replaced module directories that take precedence over the module versions that would normally be resolved by MVS, when working in any of those modules.

#26640 cmd/go: allow go.mod.local to contain replace/exclude lines

The issue of maintaining user-specific replaces in go.mod files was brought up in #26640. It proposes an alternative go.mod.local file so that local changes to the go.mod file could be made adding replaces without needing to risk local changes being committed in go.mod itself. The go.work file provides users a place to put many of the local changes that would be put in the proposed go.mod.local file.

[mvdan]

Thanks for working on this, Michael! It's clear that many people want a solution to multi-module development, given all the activity in various issues over the last couple of years.

For those who just arrived here, note that this was discussed over the previous week on the golang-tools list, too. Some points raised there resulted in edits to the proposal draft.

Overall, I really like the idea. Below are some thoughts. As a summary, I think the proposal is trying to fix a few too many issues at once.

When doing a build operation under workspace mode the go command will try to find a go.mod file. If a go.mod file is found, its containing directory must be declared with a directory directive in the go.work file.

I still wonder why we need directory (...) to be an explicit list. Using a module in workspace mode that isn't listed in go.work is an error, so why not just make directory implicitly mean all modules under the directory where the go.work file is located? Akin to how go.mod implicitly includes all packages under its directory (with some basic exception rules).

If the answer is "it's too expensive to find all modules every time", I'd love to see some proof of that. I'd also like to see if caching could be a solution, similar to go list ./.... I know you mentioned alternatives like directory ./... or go work init filling in the directory list, but either of those seems unnecessary if we can implicitly do the right thing.

go.work files allow users to operate in directories outside of any modules but still use the workspace build list. This makes it easy for users to have a GOPATH-like user experience by placing a go.work file in their home directory linking their modules together.

We touched on this in the email list, and you added this section, so as a follow-up - I'm still not convinced that the first version of go.work needs to support doing builds outside of any of the modules. Do we have any particular use cases in mind other than "be more like GOPATH"? For example, before modules I'd never run go build all or go test all from the root of my GOPATH, and I similarly do not expect to do it in workspace mode.

And if I want to build or test one of the modules, I'd want to cd into that module anyway, to make sure I'm using its go.mod file - replace directives, etc.

#39005 proposal: cmd/go: introduce a build configurations file

This issue proposes to add a mechanism to specify configurations for builds, such as build tags. This might be something that can be added in a future extension of go.work files.

I don't think that would be a good idea, to be honest. I'm starting to believe that build configurations (e.g. what build tags to use in go build) aren't only useful at the module level, and by extension not only at the workspace level either. Maybe a Go module is entirely portable by default, but one of its packages explicitly targets js/wasm only. The build config should separate the package. Similarly, when working across many modules, it's reasonable for them to fall under different build configurations.

A "build config" also has a purpose, at least when proposed in #39005: for the original author to announce to the users what are the supported ways to build their packages. Without that information, one can only assume that all build tag and GOOS/GOARCH combinations are possible, which quickly explodes exponentially. Since most users would not publish go.work in a VCS repository, the usefulness of a "build config" to give information to users is lost.

All in all, I don't think this proposal should ever aim to solve the "build config" problem.

The go.work file provides users a place to put many of the local changes that would be put in the proposed go.mod.local file.

This one is an interesting thought. This only works if we declare that go.work files should not be committed into VCS, though, and the proposal currently does not say that.

For example, if we're OK with monorepos publishing go.work files, how can that work as a place to put local changes?

[ohir]

Note: comments are related to the proposal document. Please take all "must", "should", "*-not" placed below as prepended with a humble IMO.)

If -workfile is set to off, workspace mode will be disabled. If it is auto (the default), workspace mode will be enabled if a file named go.work is found in the current directory (or any of its parent directories), and disabled otherwise. If -workfile names a path to an existing file that ends in .work, workspace mode will be enabled.

Workspace mode should NOT be turned on without a human demanding so. Proposed defaults with "auto" and go.work presence make for traps if there is go.work present in vcs but we want to build/test in module mode — we may forgot to add a flag "workspace=off" using tools by hand. Or there was an uncommited go.work we forgot to copy, or we checked out older revision, then we think we continue within workspace while we now do not. Former trap might bite the most when we'd set to work deep into the tree but go.work still hangs two floors above us: we might think we're testing this module in module mode (oh, no go.work here) while we are testing this module with dependencies replaced by an active go.work directive.

Make intent explicit either by mandating option to always be present. or by environment.
Allow user to decide on "by presence" default eg. with GOWORK="."

---

The directory directive takes an absolute or relative path to a directory containing a go.mod file as an argument.

The go.work file should not take absolute path in the directory directive — only paths relative to the place of the go.work file. Otherwise having go.work commited to the repository freezes filesystem layout at a go.work mandated state, as "no restriction on where the directory is located" is proposed too. ("No restriction, anywhere" in practice says that after a longer while of hiatus user might be missing pieces of her workspace; with no friendly go get to rescue.)

---

The replace directives in the go.work are applied in addition to and with higher precedence than replaces in the workspace modules. A replace directive in the go.work file overrides replace directives in workspace modules applying to the same module or module version. If two or more workspace modules replace the same module or module version with different module versions or directories, and there is not an overriding replace in the go.work file, the go command will report an error.

To my understanding, the ultimate goal of "workspace" is to produce versioned code for reproductible builds, hence at least the unversioned replacements should be disallowed in go.mods of modules we currently work on in the "workspace". Allowing for two or more sets of replacements overlayed would be a costly mistake.

IOW: either we commit to progress in work with set of go.mod replacements (current state), or we commit to work using workspaces and put all our replacements in go.work. Tools should guard us from the mix.

In my imagination, the "workspace" workflow (ie. workspace=on) should be a simple loop:

1. make code work;
2. test locally; if test != ok {; goto 1 };
3. tag changes and fix go.mods accordingly
4. push tagged changes upstream for CI to consume from modules proxy/product repo;

After 4 the CI pipeline should use the same code we tagged at 3!

If code to be built with workspace "=on" and "=off" does not converge at 4, we've lost the track of what we're shipping. Tools should somehow prevent that. A bare minimum is to disallow unversioned replacements in go.mods of modules pulled into workspace. The real shield would be to move all replacements from go.mod to go.work — hardly possible now.

[matloob]

@mvdan, Thanks for the feedback! A few replies:

I still wonder why we need directory (...) to be an explicit list. Using a module in workspace mode that isn't listed in go.work is an error, so why not just make directory implicitly mean all modules under the directory where the go.work file is located? Akin to how go.mod implicitly includes all packages under its directory (with some basic exception rules).

If the answer is "it's too expensive to find all modules every time", I'd love to see some proof of that. I'd also like to see if caching could be a solution, similar to go list ./.... I know you mentioned alternatives like directory ./... or go work init filling in the directory list, but either of those seems unnecessary if we can implicitly do the right thing.

I'm not convinced that the right behavior is to add all the modules under the directory, even if it were free: if we added all the modules under the directory, then it would be much harder (I think impossible?) to build a workspace that includes a module, but not any of the modules contained in subdirectories. So if we added all the modules in a directory we'd have to add another mechanism to remove modules from the workspace. And if we're going to do that it seems to me that the default should be to add a module at a time, and maybe consider if we want to do a directory tree in the future, if it turns out to be necessary.

I think the name of the directory directive is unfortunately misleading, because it does seem to imply that we're adding all the modules under a directory. But it's the best name I could come up with: unfortunately module is already taken by the go.mod file to have a different meaning.

We touched on this in the email list, and you added this section, so as a follow-up - I'm still not convinced that the first version of go.work needs to support doing builds outside of any of the modules. Do we have any particular use cases in mind other than "be more like GOPATH"? For example, before modules I'd never run go build all or go test all from the root of my GOPATH, and I similarly do not expect to do it in workspace mode.

And if I want to build or test one of the modules, I'd want to cd into that module anyway, to make sure I'm using its go.mod file - replace directives, etc.

I'm wondering what we'd get by disabling builds outside any of the modules? It's something that essentially comes for 'free' from the rest of the design because the go.work file already has to explicitly specify its set of directories, and the build list does not depend on the 'current' module. And keeping this in reinforces the point that if you are in the same workspace
cding into the module doesn't change go.mod file or the set of replace directives that are used

I don't think that would be a good idea, to be honest. I'm starting to believe that build configurations (e.g. what build tags to use in go build) aren't only useful at the module level, and by extension not only at the workspace level either. Maybe a Go module is entirely portable by default, but one of its packages explicitly targets js/wasm only. The build config should separate the package. Similarly, when working across many modules, it's reasonable for them to fall under different build configurations.

A "build config" also has a purpose, at least when proposed in #39005: for the original author to announce to the users what are the supported ways to build their packages. Without that information, one can only assume that all build tag and GOOS/GOARCH combinations are possible, which quickly explodes exponentially. Since most users would not publish go.work in a VCS repository, the usefulness of a "build config" to give information to users is lost.

All in all, I don't think this proposal should ever aim to solve the "build config" problem.

I think that the workspace file could be useful to allow the developer to specify configuration to tooling, because it is developer specific configuration, but this proposal does not intend to solve the build config problem. It's definitely out of scope for the design. If it's too distracting for the conversation I can remove the reference from the related issues section in the doc.

This one is an interesting thought. This only works if we declare that go.work files should not be committed into VCS, though, and the proposal currently does not say that.

For example, if we're OK with monorepos publishing go.work files, how can that work as a place to put local changes?

The doc does recommend that "most go.work files should exist outside of any repository" and I do think that go.work files in should rarely be checked into VCS. (Though it's buried in one paragraph in the doc so maybe I could do a better job of specifying that.)

If a monorepo publishes a go.work file, a user can still put a go.work file in a parent directory of the repository and run the go command from there. This is another one of the benefits of allowing workspace builds outside of any particular module. It allows there to be multiple workspace 'directories' that a particular module can participate in without needing to pass -workfile. Of course a user can also pass in -workfile, either directly or through GOFLAGS.

[matloob]

@ohir, thanks for the comments!

Workspace mode should NOT be turned on without a human demanding so.

I definitely understand the issue about users not realizing which workspace they're in and accidentally ending up with the wrong build. I think this is most likely to happen if the workspace file is checked into a repository. (Otherwise, the user created the workspace file themselves, and opted-in to workspace mode). Would you agree with that?

I think in general checking in a workspace file to a repository should be rare: because it is really only useful for multi-module repositories which themselves are not the common case, and because it will affect the build configuration for every developer of a module, it's something that should only be done if it fits into the workflow of most developers.

The go.work file should not take absolute path in the directory directive — only paths relative to the place of the go.work file.

Do you think this would be a problem for go.work files that are not checked into a repository? Of course, I agree that absolute paths should not show up in a go.work file that's checked into a repository (for that matter, there shouldn't be any relative paths that point outside the repository). If such a change was committed, I expect that it would be quickly reverted because it likely wouldn't work for any other developers than the one who originally committed it!

Here is a case for absolute paths: some editors might want to make temporary workspace files because a user just opened two files in different modules. In that case the workspace file would be put in a temporary directory, and it would make more sense to add absolute paths rather than trying to construct a relative path from the temporary directory to the location of the module. And on Windows, I believe it's not possible to construct relative paths across volumes, so the editor would have to attempt to make a temporary directory on the same volume, something that might not be easy to do.

I wonder if this is something an outside tool can catch, like a sort of workspace vet command?

To my understanding, the ultimate goal of "workspace" is to produce versioned code for reproductible builds, hence at least the unversioned replacements should be disallowed in go.mods of modules we currently work on in the "workspace". Allowing for two or more sets of replacements overlayed would be a costly mistake.

Oh! I think the doc must have not been as clear: the ultimate goal is not to produced versioned code for reproducible builds, but to assist in workflows on multiple modules. To get the reproducible build you still need a single go.mod file that a user will build from. I wonder if you have ideas for how to make the intention more clear in the document.

Because replacements are made for developing modules, I think users would be surprised to not have them in workspaces. I think it's a bit unfortunate that they need to be supported in workspaces, but it's hard for me to see a better solution.

If code to be built with workspace "=on" and "=off" does not converge at 4, we've lost the track of what we're shipping. Tools should somehow prevent that. A bare minimum is to disallow unversioned replacements in go.mods of modules pulled into workspace. The real shield would be to move all replacements from go.mod to go.work — hardly possible now.

I think having replaces in multiple places is a good idea, but (unless I'm misunderstanding) this seems to encourage users to check in go.work files in single module workspaces, which could impede users setting up their own workspaces (see my reply to @mvdan )

[mvdan]

if we added all the modules under the directory, then it would be much harder (I think impossible?) to build a workspace that includes a module, but not any of the modules contained in subdirectories

I see. I can't say I can think of any scenario where I'd need this, but I've also avoided nested Go modules at any cost.

it's the best name I could come up with

How about include, import, or just modules?

I'm wondering what we'd get by disabling builds outside any of the modules?

I'm looking at this from the other side; why should we support this edge case right away if we don't have a strong use case for it? Doing builds inside the workspace but outside any module is something we could add at a later time if users really ask for it, but we wouldn't be able to take it back if we later find out it's confusing or unnecessary.

I think that the workspace file could be useful to allow the developer to specify configuration to tooling, because it is developer specific configuration, but this proposal does not intend to solve the build config problem. It's definitely out of scope for the design. If it's too distracting for the conversation I can remove the reference from the related issues section in the doc.

I'd remove that section, personally. Nowadays I tend to think that build config shouldn't need to be attached at the module level, and the same applies to the workspace level.

The doc does recommend that "most go.work files should exist outside of any repository" and I do think that go.work files in should rarely be checked into VCS. (Though it's buried in one paragraph in the doc so maybe I could do a better job of specifying that.)

I think making that recommendation clearer is a good idea, as it's probably one of the first questions users will have.

If a monorepo publishes a go.work file, a user can still put a go.work file in a parent directory of the repository and run the go command from there. This is another one of the benefits of allowing workspace builds outside of any particular module. It allows there to be multiple workspace 'directories' that a particular module can participate in without needing to pass -workfile. Of course a user can also pass in -workfile, either directly or through GOFLAGS.

Hmm, I find that recommendation for users to be a tad weird. We're introducing nested workspaces and builds outside any module, all for the sake of being able to satisfy the use case of local-only replace directives. It feels like a bit of a stretch :)

[neild]

The presence of a go.work file in the working directory or a containing directory will put the go command into workspace mode.

Does this imply that every go command invocation which reads a go.mod needs to check for go.work in every directory up to the fs root?

I’ve worked on systems with slow network filesystems where that would be expensive.

[matloob]

it's the best name I could come up with

How about include, import, or just modules?

include could work. Added it as an option in the "Open issues" section

I'm wondering what we'd get by disabling builds outside any of the modules?

I'm looking at this from the other side; why should we support this edge case right away if we don't have a strong use case for it? Doing builds inside the workspace but outside any module is something we could add at a later time if users really ask for it, but we wouldn't be able to take it back if we later find out it's confusing or unnecessary.

I really don't think of this as an edge case, it's just a simple way to determine how to set the build list: "Prefer workspace mode (ie search for a go.work), if found use its build list. If not found, use module mode (ie search for a go.mod), and use its build list". I think we'd need to have a stronger reason to add the extra case of making an extra rule to disable it, for example cases that it could become confusing for.

I think that the workspace file could be useful to allow the developer to specify configuration to tooling, because it is developer specific configuration, but this proposal does not intend to solve the build config problem. It's definitely out of scope for the design. If it's too distracting for the conversation I can remove the reference from the related issues section in the doc.

I'd remove that section, personally. Nowadays I tend to think that build config shouldn't need to be attached at the module level, and the same applies to the workspace level.

Got it. I'll start by make it it much more clear that we have no intention of addressing that use case in this proposal or extending go.work in the future to address it.

The doc does recommend that "most go.work files should exist outside of any repository" and I do think that go.work files in should rarely be checked into VCS. (Though it's buried in one paragraph in the doc so maybe I could do a better job of specifying that.)

I think making that recommendation clearer is a good idea, as it's probably one of the first questions users will have.

Done, in the latest revision, and strengthened it to advocate against checking in go.work in vcs period.

If a monorepo publishes a go.work file, a user can still put a go.work file in a parent directory of the repository and run the go command from there. This is another one of the benefits of allowing workspace builds outside of any particular module. It allows there to be multiple workspace 'directories' that a particular module can participate in without needing to pass -workfile. Of course a user can also pass in -workfile, either directly or through GOFLAGS.

Hmm, I find that recommendation for users to be a tad weird. We're introducing nested workspaces and builds outside any module, all for the sake of being able to satisfy the use case of local-only replace directives. It feels like a bit of a stretch :)

Thinking about this much more in light of your comments and @ohir's I think recommending go.work files to be checked in to repos was a mistake so I'm going back and uniformly recommending against it.

[meling]

Caveat: I've skimmed the proposal but haven't really tried to understand all the nuances here, so this may well be too simplistic.

My suggestion would be to try really, really hard to do this without introducing the go.work file. Here is an idea that maybe could work, using only go.mod with some minor additions.

Here workspace is a magic module name that the go command can interpret in a similar way to the go.work file.

module workspace

go 1.18

require (
	./baz	foo.org/bar/baz
	./tools	golang.org/x/tools
)

If the go.mod file for some module has require entries with a local path, these would be used; I guess this would be similar to a replace entry.

module github.com/relab/gorums

go 1.18

require (
	./baz	foo.org/bar/baz
	./tools	golang.org/x/tools
	google.golang.org/grpc v1.36.1
	google.golang.org/protobuf v1.26.0
)

As I mentioned at the top, I don't know if this would work for all scenarios, but I would strongly encourage finding a simple design.

Obviously, I understand adding another field to the require entries makes parsing a bit more challenging. Still, I think there are reasonable solutions to this, e.g., paths must start with ./ or ../ or /. Alternatively, one could add a => between the path and the module name, if necessary.

[matloob]

The presence of a go.work file in the working directory or a containing directory will put the go command into workspace mode.

Does this imply that every go command invocation which reads a go.mod needs to check for go.work in every directory up to the fs root?

I’ve worked on systems with slow network filesystems where that would be expensive.

I want to understand this a bit better: my assumption for these cases are that most builds (or lists other go command operations that need the info in a go.mod will need to open many files and directories, and the lookups done for go.work wouldn't dominate. Is that not the case? Are there operations or filesystem layouts that would be particularly bad here?

[neild]

I want to understand this a bit better: my assumption for these cases are that most builds (or lists other go command operations that need the info in a go.mod will need to open many files and directories, and the lookups done for go.work wouldn't dominate. Is that not the case? Are there operations or filesystem layouts that would be particularly bad here?

I've been on systems where stats on files under $HOME were reasonably fast, but $HOME/.. was slow due to being a network automount. I recall AFS as being one offender here, although it's been quite a while and perhaps I'm unfairly maligning it.

Perhaps this isn't a case that matters any more these days; multiuser systems are certainly out of vogue.

[matloob]

@meling I'd like to understand better why you'd like to avoid introducing the go.work file? I think making a kind of magic go.mod file instead could be more confusing and make things more complicated.

I don't totally understand the second example: what does it mean when you have both types of requires in a 'regular' module?