x/all: start tagging versions

[leonklingele]

With the advent of tool like dep it makes sense to tag a specified release instead of referring to master or a specific commit.

[ALTree]

It would be useful to have a common policy for all subrepos (or at least the ones for which it makes sense to publish "releases"), not just crypto.

[rsc]

This discussion should be about all the subrepos, not just x/crypto.

[rsc]

There is also a question of how to decide the right version number, when to bump major version, what tagging implies (any particular testing), and so on. There are many issues here.

[rsc]

@mpvl was starting to look into this for x/text, which might be a good starting point.

[ianlancetaylor]

One step is that we need to figure out how to split up the x/ repos into modules.

[bradfitz]

On hold until Go 1.11 and/or Go 1.12. At least until we have tooling ready and some experience with said tooling.

[networkimprov]

As a start, just maintain a semver tag for each package in golang.org/x/* (or at least in the mostly-stable repos), since a package is the unit of use in most cases. The natural starting semver is 1.0.0 (unless a package is considered unstable, in which case 0.1.0).

For now, let users create modules from tagged packages as needed. Eventually you may decide to bless certain of those modules.

You might want to add tags ASAP; the imminent rush of go mod users will complain if they have to ref commit IDs :-)

@gopherbot add modules

[thepudds]

Related to #27858 and #28136.

[psampaz]

any updates in this? Is it still valid for this proposal to be on hold?

[itsmontoya]

This is creating a lot of noise within go.sum files when many libraries utilize x/net and x/sys. In addition, it's creating difficulties when using go plugins due to differing versions (usually caused by x/net and x/sys). Do we have an update on when this is going to get implemented?

[ALTree]

@bradfitz wrote in 2018:

On hold until Go 1.11 and/or Go 1.12. At least until we have tooling ready and some experience with said tooling.

Now Go1.14 is out and we have a some experience on the module workflow, so I'm moving this from "on hold" to "Proposal" for re-evaluation.

[networkimprov]

The tag doesn't need to be a contrived x.y.z string. It could be 0.0.0-timestamp, posted after each new feature/fix.

[bcmills]

@networkimprov, isn't that essentially what we have with pseudo-versions today? (Or do you mean we should tag one of those after each significant change only?)

[networkimprov]

Depends what you mean by "significant" but probably that; I didn't mean after each commit.
0.0.0-timestamp is a lot more meaningful than 0.0.0-SHA

You could limit the frequency to once a week/month/quarter etc.

[itsmontoya]

I was asked by @bradfitz to think about this issue a bit more:

Please take this to the issue. Describe in your post there how you think tagging the x/foo repos helps the plugin situation. Imagine as a thought experiment we auto-tagged on each commit. How's that different than today?

The biggest difference is the proper versioning will allow for all the go.mod related tooling/plugins to more easily keep versioning up to date. In my experience, whenever a library is un-versioned - none of my IDEs seem to know or care about an update (if and when it's available). In the world of a versioned library - as soon as the new version is published, my IDE instantly notifies me and asks if I would like to upgrade the library. As a result, my codebase stays up to date with the library on the parent repository. If Go's gomod tooling supported non-versioned libraries as first class citizens, and aided the maintainer in keeping the library up-to-date - we would experience a lot less fragmentation.

Now this thought experiment is interesting, and allows us to see the true benefits of the change (or at least ways we could improve the process if the change doesn't occur). Outside of the thought experiment - it is my opinion that golang.org/x/ packages being properly versioned will encourage adoption of versioning as a best practice. If those practices are modeled by the system libraries and demonstrated by system tool defaults, the usage of proper versioning should increase.

My hope is that we can move forward with applying basic versioning to all golang/x/ repos, and start the process of eliminating some of the serious version discrepancies we've been experiencing. As a parting note, I am including a link to goacme/lego's go.sum file. This is a fantastic library which provides Go services with an easy way to implement Let'sEncrypt. As you can see, this project is seriously plagued with version discrepancy issues.

https://github.com/go-acme/lego/blob/master/go.sum

[bcmills]

@itsmontoya

As a parting note, I am including a link to goacme/lego's go.sum file. … As you can see, this project is seriously plagued with version discrepancy issues.

A large go.sum file is not a symptom of “version discrepancy issues”, but rather of a transitive dependency graph containing many lower-than-selected versions. But we are already working on #36460, which will hopefully eliminate the need to load the vast majority of these otherwise-unused dependency versions. Given that, I don't think the number of possible versions itself is a compelling argument one way or the other.

That said, I agree that tagging specific releases would help users know when the libraries settle into a state in which any risky changes (or prospective API changes) have been resolved, and to avoid depending on immature features that are likely to need further revision.

[xdg]

A specific example of the kind of tooling this would support is dependabot for automated PR generation when new dependency versions are released.

[hyangah]

This was already implemented - With the work for #48523, many golang.org/x repos are getting automatically&periodically tagged.

EDIT: OTOH I don't think there is officially announced versioning policy. My understanding is that most repos will remain in v0, and the tagging process will keep incrementing the minor version. (point release is for a special release to fix critical issues is necessary) Since they are on v0.1.0, strictly speaking, there is no API stability guarantee - even though Go team works hard to avoid breakage.

[rsc]

Removed from proposal process and closing. @heschi and others have implemented a first cut at this. It may require some fine-tuning, but the only way to learn is to do it and see. Right now the plan is to tag x repos that are passing all tests on all builders approximately monthly. Part of the tagging of any given repo is first updating its go.mod to refer to the most recently tagged versions of other repos (the repos are processed dependencies first).

This does mean that if you want the very latest (possibly broken!) version of an x repo, you need to use 'go get golang.org/x/foo@master' (or '@commithash') instead of just 'go get golang.org/x/foo'. This was already true for x/text and x/tools, although we were not good about remembering to apply new tags to those. Now the tagging is automated and won't rely on our memories.

There will also be a way for us to tag a given repo out of cycle if needed, but we should expect that most changes just go out in the usual automated tag and people who really need a change earlier use 'go get golang.org/x/foo@master'.