Go versioning: add support for maintaining multiple versions within a single repository

[galargh]

Extract from a conversation with @masih:

There are some multi-module repositories that maintain multiple versions on the same branch. The ask here is to handle both root level version.json file and version.json files from subdirectories if they exist.

One could update either of the version.json files for the new tag to be created. Maintainers are aware that tags can’t clash.

Example repo that would benefit from such setup:
https://github.com/ipld/go-car
https://github.com/ipld/go-car/tree/master/v2

Alternatively: maybe the root level version.json could be structured in a way that would support multiple versions.

[mvdan]

Worth noting that, at least as far as Go is concerned, tags apply to modules - not repositories. So, intuitively, I would expect one version.json next to each go.mod file that we release publicly.

That vision is somewhat Go-centric though. If we prefer always a top-level version.json, with a more complex structure when there are modules in subdirectories, that would also work.

[masih]

My vote is version file next to go.mod please

[mvdan]

Assuming that versioning will be following the language in question, I think I do agree with version.json next to go.mod. You could imagine the same idea for other languages - version.json next to package.json and so on. There's no hard requirement that the package be at the root level of the repository, or that there be only one package per repository, or even that all packages in a repository share the same releases. go-car is a good example of the last two edge cases.

[vmx]

A note from the Rust world. You can also have multiple packages (called crates) within a single repository. There is a root Cargo.toml, which defines that it is a workspace, with individual crates. Each crate will then have its own Cargo.toml with the corresponding meta information. There it would also make sense to have a version.json next to each crate's Cargo.toml.

[marten-seemann]

Putting a version.json next to every go.mod shouldn’t be the problem, but I’m having trouble understanding how this works at the level of Git. After all, tags are per-repo, as Git is agnostic regarding the repo contents.
How does Go “know” which tag belongs where?

Am I right to assume that if you have multiple versions (like go-car for v1 and go-car/v2 vor v2) in the same repo, Go will just match all v1.x tags to go-car and all v2.x tags to go-car/v2?
What happens if the separate module is examples/, as we have in go-libp2p. Which version tag would apply to that package?

[masih]

Am I right to assume that if you have multiple versions (like go-car for v1 and go-car/v2 vor v2) in the same repo, Go will just match all v1.x tags to go-car and all v2.x tags to go-car/v2?

Yes the major version compatibility is checked. You can try go get github.com/ipld/go-car@v2.1.1 and you'll get:

go get: github.com/ipld/go-car@v2.1.1: invalid version: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v2

[mvdan]

@marten-seemann your understanding is about right. There are four basic rules:

1. semver tags can only apply to a module that agrees with the major version (e.g. a semver v0 or v1 can only apply to module foo/bar, not foo/bar/v2).
2. regular semver tags of the form v1.2.3 can only apply to the module (go.mod) at the root of the repository
3. "prefixed" semver tags like foo/bar/v1.2.3 can only apply to the module under foo/bar in the repository
4. as an exception to rules 2 and 3, you are allowed to maintain a v2+ module in a vN subdirectory instead of a branch, and you can use tags like v2.3.4 without needing a prefix like v2/v2.3.4. this is what go-car/v2 uses.

More details in https://go.dev/ref/mod#vcs-version and the rest of that doc. But hopefully my summary helps get a high-level picture. I know the rules seem complex and arbitrary, but you need all of this to support all reasonable workflows - including monorepos - without forcing all modules inside the same repository to always share all versions.

[marten-seemann]

@mvdan Thank you for the detailed explanation!

3. "prefixed" semver tags like foo/bar/v1.2.3 can only apply to the module under foo/bar in the repository

Interesting. I didn’t know that tags like these were even possible (or rather, valid semver tags). We’ll need to modify the workflows such that they don’t only trigger on v*.

[mvdan]

I think that's been fine so far, because the so-called "nested Go modules" are generally discouraged unless you're going for a proper monorepo. They are rare at PL, given that practically nothing uses monorepos at the moment.

There are some examples of nested modules, such as go-car/cmd and go-ipld-prime/storage/bsadapter, but note that noone has released tagged versions for those yet. If they had, they would have beem of the form cmd/v1.2.3 or storage/bsadapter/v1.2.3.

We’ll need to modify the workflows such that they don’t only trigger on v*.

That sounds right. You want to allow an optional "subpath prefix" as well. But then we'll also need to properly navigate into the nested module to look at the right go.mod file (and possibly its separate version.json file).

[galargh]

This issue was transferred to ipdxco/unified-github-workflows#40 in preparation for archiving of this repository.