Ignore nested workspaces

[JordanMartinez]

Below is my understanding of this request based on the corresponding conversation in Discord. https://discord.com/channels/864614189094928394/865401625249448006/1084363652238413955

A monorepo is a group of packages that can compile together via some master build. A multirepo/polyrepo is a group of packages that cannot be built together; rather, they are just stored together. For context, see https://monorepo.tools/.

In the new spago, a workspace is defined by some folder that contans a spago.yml file. For example, this repo structure is a monorepo:

foo/Foo.purs
bar/Bar.purs
baz/Baz.purs
spago.yml

However, this repo structure is a multirepo:

client/
  Client.purs
  spago.yml -- multirepo package - uses JS backend
server/
  Server.purs
  spago.yml -- multirepo package - uses Erlang backend
shared/
  Shared.purs
spago.yml -- Root workspace - backend depends on usage

When building the root spago.yml, should it also consider things found in the server/spago.yml? There are a few ways to resolve this issue:

1. Throw an error - disable this entirely.
2. Use the workspace defined in the subdirectory? - use the root for root things and the server for server things
3. Merge the two workspaces?
4. Ignore the nested workspace

The answers are:

1. No. Throwing an error shouldn't be done so that multirepos can be supported.
2. No. But it's not clear to me why. Perhaps because one doesn't always want to build server when building shared?
3. No. Merging the two shouldn't be done because different backends may be used between workspace and it prevents a situation where one workspace uses version A of some package and another workspace version B of the same package. PureScript doesn't support compiling two different versions of the same package as it affects type class resolution and whatnot.
4. Yes.

See https://github.com/purescript/spago/blob/master/spaghetto/src/Spago/Config.purs#L154-L155 for where this todo originated.

[considerate]

2. No. But it's not clear to me why. Perhaps because one doesn't always want to build server when building shared?

The reason I'm inferring is that it's because you can only know that packages built with the same compiler and backend work together. Hence, if you have the following directory structure and package-b declares a different workspace than the client workspace spago.yaml it's not obvious what compiler to use when compiling package-a assuming package-a is a dependency of package-b.

• If you use the compiler specified in the client workspace you can't be sure it will work with package-b.
• If you use the compiler in the workspace given by package-b it will work but it might be confusing. Also, now if you compile package-a directly you end up with two versions of package-a, one compiled in the package-b workspace and one in the client workspace.

.
└── client
    ├── package-a
    │   └── spago.yaml # package
    ├── package-b
    │   └── spago.yaml # package, workspace
    └── spago.yaml # workspace

[f-f]

Thanks for opening this Jordan!

A few clarifications:

For example, this repo structure is a monorepo

That repo structure is not a monorepo, but a single repo, since there's only one configuration file.

However, this repo structure is a multirepo

The directory structure itself doesn't tell you directly if something is a monorepo or a multirepo. It's a multirepo if there are independent builds to compile all the sections.
But if there are any shared packages, then it's a monorepo.

In the "base case" of a "spago monorepo", the root configuration contains a workspace section and the nested ones don't.
If a nested configuration contains a workspace section, then we should ignore it, together with its whole subtree. This means that you'll need a separate build inside that workspace to compile it, since the root build won't - note that this doesn't automatically make it a multirepo, since it can still be a monorepo if you import packages from the root scope (you still can, but it doesn't happen automatically anymore as it's the case of nested configs).

No. But it's not clear to me why.

As @considerate noted, it's not possible to mix different workspaces in the same build, as they might specify different package sets, backends, etc. Everything ends up in the same output folder and compiler versions/package sets need to match.

[considerate]

Personally I think the saner default is to throw an error and add an --exclude flag to exclude the nested workspaces. Alternatively, ignore the nested workspaces but listing all excluded packages in a warning message.

Silently ignoring the nested workspace will probably make sense after getting used to that behavior but it's likely to be non-intuitive to many at first.

[f-f]

Spago should throw errors for unrecoverable situations, and this is not one of them - this is just part of the normal autodiscovery mechanism for packages in a monorepo. This behaviour, together with the behaviour for excluding nested workspaces that we're discussing here, mirrors how Bazel treats workspaces.

Discovered packages will also be printed on every execution so that the user can see if the package they need is in the list. Excluded packages are currently not printed by default, but end up in debug logs, and I think it should stay like that, or we'd risk unnecessarily polluting the output with unnecessary information (because if a user puts together a situation with nested workspaces, they probably want them to be built separately, so there's no point in printing a big warning "look, these nested packages have been excluded")