Proposal: go/list adds another flag to support showing major upgrade available for module

[zachgersh]

⚠️ DEPRECATED ⚠️

Please see the linked issue that has replaced this one.

Proposal

Currently, with SIV there isn't an automated way to understand that you may want to upgrade a package to a new major.

For example, if I have a mod that looks like this:

module github.com/test/test

go 1.14

require (
	github.com/peterbourgon/ff v1.7.0 // indirect
	rsc.io/sampler v1.2.1 //indirect
)

and I use go list like so:

go list -m -u all

I am going to get output like this:

github.com/test/test
github.com/BurntSushi/toml v0.3.1
github.com/davecgh/go-spew v1.1.1
github.com/mitchellh/go-wordwrap v1.0.0
github.com/pelletier/go-toml v1.6.0 [v1.7.0]
github.com/peterbourgon/ff v1.7.0
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c [v0.3.2]
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7 [v0.0.0-20191204190536-9bdfabe68543]
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 [v1.0.0-20200227125254-8fa46927fb4f]
gopkg.in/yaml.v2 v2.2.4 [v2.2.8]
rsc.io/sampler v1.2.1 [v1.99.99]

Now github.com/peterbourgon/ff as a module actually has a v2 and a v3 and I would like go list to tell me of their existence sort of like pkg.go.dev does now when I visit the versions page:

Adding a flag to go list seems like a good step to signal to a user that they may want to upgrade their package to a newer major.

[bcmills]

The presence of a new major version does not necessarily imply lack of support for the previous version.

For example, if /v2 is implemented in terms of /v3, then there is no pressing need for users to “upgrade”, as the previous API can remain supported indefinitely. (And, in that case, the upgrade is properly a task for an automated refactoring tool (such as proposed in #32816) instead of go list: it's more a matter of code cleanup than “upgrading” per se.)

On the other hand, if a package author intends to signal that part or all of a particular API is unsupported and cannot be expressed as a wrapper around a supported API, they can already mark its contents Deprecated to discourage new uses, and the deprecation comment can refer to the supported replacement explicitly. (So go list is probably still not the right place to surface this information, because it would be redundant: the user would need to read the documentation in order to understand the non-trivial migration path anyway, and tools such as staticcheck already display warnings about use of deprecated APIs.)

CC @jayconrod @matloob @ianthehat @marwan-at-work @myitcv @stamblerre

[marwan-at-work]

I agree with the above. However, the Go command should still provide a way to tell a user that a certain package has newer major versions. There two important use cases for this:

1. If you are trying to go get a package, and you don't realize there's a /vN suffix, you could be a few major versions behind. I argue that "go getters" should try their best to read package documentation before running a go get, but I think it's still a valid use case.

2. You just might want to be able to list all major versions for a given module. Yes, to the machine, they are technically separate modules and can be even imported together. But to humans, it can be quite useful for many reasons such as discoverability, tooling/analysis or documentation. pkg.go.dev for example already does this (not sure how yet as it's still closed source): https://pkg.go.dev/github.com/InVisionApp/go-health?tab=versions

Therefore, I don't think upgrading from vN to vN+M is the only reason why this feature might be important. For the two reasons above, I can't think of another place other than go list but I don't feel too strongly about it.

[stamblerre]

gopls also shows suggestions for the user to upgrade to the latest version of a module. #38339 is tracking work on improving this behavior to be more user-friendly.

[zachgersh]

@stamblerre - this is really awesome, I am definitely going to keeping checking in on this!

@marwan-at-work - I agree with both of your points. I don't necessarily know whether go list is 100% the right place for this but it felt right for the time being. I also want to point out that it is my hope that surfacing these major versions (we don't even need to show their patch levels) would encourage user to read the docs and then decide if they wanted to upgrade. It's more of a nudge.

@bcmills - thanks for bringing in all of these other folks! RE: Deprecated I've not seen a project actually use this flag and I think its asking quite a bit of authors when technically a package could be seen as Deprecated when they stop providing minor + patch updates. I want the tooling to signal to users "hey you could upgrade" and push them toward reading the docs. Honestly, most users in their day-to-day workflow need all the help they can get when it comes to versions of packages.

[bcmills]

I think its asking quite a bit of authors when technically a package could be seen as Deprecated when they stop providing minor + patch updates

A project could quite reasonably stop providing minor + patch updates without deprecating the API, simply because they are not making further changes to the old API and have no important open bugs to fix. (For example, I would not be surprised if github.com/golang/protobuf reaches that state very soon, because ~all of the improvements will be in the underlying implementation at google.golang.org/protobuf.)

Moreover, there is no guarantee that a maintainer will stop providing minor + patch updates to v1 just because they have released v2. (Many maintainers backport critical fixes to one or more previous versions, as the Go project itself does with Go releases.)

So it really is important to explicitly deprecate a package when it is no longer suitable for new usage.

[bcmills]

I want the tooling to signal to users "hey you could upgrade" and push them toward reading the docs.

Maybe? But we don't want to induce “upgrade fatigue” by signaling upgrades that aren't actually needed, especially when they require active intervention (as implied by a differing major version).

“You could upgrade” is very different from “you should upgrade”: using an older version of that API is fine if that API is maintained, but dangerous if that API is no longer receiving security fixes.

[zachgersh]

Maybe? But we don't want to induce “upgrade fatigue” when it isn't needed.

I very much agree with this. I think having a flag in go list (that is not on by default) should make it that users are dealing with minor versions most of the time but they can run it with some other flag that then will tell them about packages that have majors available?

The proposal doesn't go into how this flag operates, I want to come to agreement on the core idea of surfacing this information to users via the go tool so that they can take whatever meaningful action they see fit.

[jayconrod]

I think a flag like this could make sense as part of a larger vision for upgrading across major versions. But I don't think we should start taking steps on that path until we know where we're going.

The problem is there's not much a developer can usefully do with the go command, knowing a new major version is available. A major version is a breaking change by definition, so some refactoring is going to be required. Tools don't yet provide much help with that.

Another question is whether the new major version can coexist with the old version. A developer may have dependencies that still require the old version, so if they upgrade, they'll have to link packages from both versions into the same project. If there's global state, that can be a problem. Ideally, the author of the new major version has taken care of that, but it's not always the case in practice.

[zachgersh]

This is now deprecated in favor of a combine proposal that is linked to this issue