Make version="v1.2.3" and version="=v1.2.3" equivalent or throw an error

[c2h5oh]

What version of Go (go version) and dep (git describe --tags) are you using?

go version go1.8.3 linux/amd64
dep v0.1.0-310-g8a46f4d

What dep command did you run?

dep ensure with Gopkg.toml containing:

[[constraint]]
  name = "upper.io/db.v3"
  version = "v3.2.2"

What did you expect to see?

version tagged with v3.2.2 in vendor OR an error message about invalid version syntax

What did you see instead?

latest version - v3.3.1 at the time I run the command - in vendor

I know that the version constraint syntax to lock to a specific version is =version (so I should have had version = "=v3.2.2" in my Gopkg.toml), but silently installing a different version is not OK. Dep should IMO either fail with invalid version error (and maybe a hint) or make the two options equivalent.

[sdboyer]

hi! i know this is something that can be confusing and frustrating, and we need to improve docs around it.

as a practical matter, we're locked into it now - changing the meaning of the symbols (so that = is implicit when no operator is present, instead of ^) would have the effect of breaking the guarantee around the stability of the Gopkg.toml and Gopkg.lock files. it's something we can revisit when it comes time to move into the toolchain.

but, we have to give it an honest shot. people expressing unnecessarily narrow constraints to serve immediate-term needs is a path that could land us all, as a community, in painful dependency hell. this mechanism is one of a set that helps nudge us in a different direction. this is a nuanced and carefully-weighed design choice that, unfortunately, really only makes sense in the big picture; i need to write a blog post laying it out.

the basic reason we believe this tradeoff is acceptable is because your Gopkg.lock should be adequate for keeping versions stable in almost all situations. to that end:

dep ensure with Gopkg.toml containing:

did a Gopkg.lock exist when you ran dep ensure? if so, and if upper.io/db.v3 was locked to v3.2.2, then dep ensure should not have upgraded you to the latest. if it did, that's almost certainly a bug.

if, OTOH, you actually ran dep ensure -update, then the advice here is to enumerate the dependencies you want to update, rather than blanket-updating all of them (which is the behavior when no additional arguments are passed).

[sdboyer]

couple more thoughts...

but silently installing a different version is not OK.

just to be clear, we fundamentally agree on this. the issue is the expectations for when that happens, and the corresponding approach taken to achieving goals.

Gopkg.toml describes what's possible - a version set your program probably works with. almost all the time, the only thing that narrowing a constraint to a single version in Gopkg.toml achieves is to make later update processes more awkward.

there certainly are cases where it make sense to aggressively restrict to only a single version of a dependency. but, most of the time, the goal folks are looking to achieve is stability for now. that is, in your case, is it true that your application only works with v3.2.2 of upper.io/db.v3? or is it that you're just not ready to update to v3.3.1 quite yet - but, if your project has any dependers, things might work fine with that version?

the former case is less common, so we give it the more awkward constraint syntax. the latter case is the overwhelming majority, so we prioritize it with an implicit ^, on the expectation that dep users will benefit most from using dep ensure -update like a scalpel.

when constraints are narrowed to a single version, dep ensure -update ceases to be useful at all, and you have to go in and make constant edits to Gopkg.toml by hand - all for very little in terms of the actual stability of your dependency versions. i realize this is what Go users are generally accustomed to, as this is how other tools have historically behaved. this workflow is different, but we think will end up being quite a bit less dreary and painful.

there are some additional flags and behaviors we could introduce to make this an even safer process, even with the open constraints, and i'm entirely open to doing that.

[VojtechVitek]

@sdboyer I'm just reading through semantic versioning examples in README.md and saw the following as an implicit behavior for version without an operator:

1.2.3 becomes the range >=1.2.3, <2.0.0
vs.
0.2.3 becomes the range >=0.2.3, <0.3.0

How come the second line is constrained on the minor range (~) and not on the major range (^)? Is the zero somehow magical? Shouldn't this be:

- 0.2.3 becomes the range >=0.2.3, <0.3.0
+ 0.2.3 becomes the range >=0.2.3, <1.0.0

?

[sdboyer]

@VojtechVitek yes, unfortunately, there's even more magic in the 0.x.x range. from the semver spec:

Major version zero (0.y.z) is for initial development. Anything may change at any time. The public API should not be considered stable.

technically, we can't assume anything about the relationship between versions in the zero range. if we were to follow that strictly, then the caret - which it is helpful to think of as a "compatibility" operator, rather than a numerical one - should be meaningless below 1.0.0.

however, other systems (cargo and pub at least), recognize that developers often (more or less) follow the semver guidelines, even below 1.0.0. from cargo's docs:

While SemVer says that there is no compatibility before 1.0.0, many programmers treat a 0.x.y release in the same way as a 1.x.y release: that is, y is incremented for bugfixes, and x is incremented for new features. As such, Cargo considers a 0.x.y and 0.x.z version, where z > y, to be compatible.

such is the algorithmic and dependency-hell-avoiding importance of having open constraints that we're willing to dance into this territory. but nuances like these are exactly why all this needs a blog post, as well as probably its own dedicated help section.

[c2h5oh]

If there was no magic implying caret for stable releases then there would be no need for a yet another, different magical behaviour for the 0.x.x versions.

If changing version="1.2.3" to be equivalent of version="=1.2.3" is not an option because if would introduce unexpected changes to existing packages I believe we should consider version="1.2.3" invalid and offer and error message explaining that it used to mean ^1.2.3, update to ^1.2.3 if you want it to mean >=1.2.3, <2.0.0 and to =1.2.3 if you want exact version match.

[sdboyer]

If there was no magic implying caret for stable releases then there would be no need for a yet another, different magical behaviour for the 0.x.x versions.

It's still just carets below 1.0.0 - no new magic.

I believe we should consider version="1.2.3" invalid and offer and error message explaining that it used to mean ^1.2.3, update to ^1.2.3 if you want it to mean >=1.2.3, <2.0.0 and to =1.2.3 if you want exact version match.

this would still invalidate old, already-committed Gopkg.toml files.

now, we could probably apply the extra validation you describe only for the current project's constraint declarations, but use the implicit caret for Gopkg.toml files in dependencies. that would honor historical intent, and without ambiguity. the transition would still be jarring, though, as we'd still be causing sudden, unexpected breakages in existing builds. that's not outside the realm of possibility.

however, every counterargument to this, both here and on the reddit thread, has focused on peoples' dislike for the narrow, immediately-obvious problem that implicit carets are surprising and non-obvious.

i don't dispute that it can be counterintuitive and frustrating when you first encounter it. i never have. but we didn't just do this to annoy people - there are larger tradeoffs to weigh here, which i've at least summarized above. the arguments i need to see to move on this must take this bigger picture into account.

[JelteF]

I'm actually fine with this behaviour, but agree it can be quite confusing. How about a warning when a bare version is used? That wouldn't break backwards compatibility.

[ibrasho]

@sdboyer Another suggestion, a bit silly, that could elevate some of the issues around this:

If we implicitly add a caret (e.g. version="1.2.3"), rewrite a new Gopkg.toml with an explicit caret (e.g. version="^1.2.3") and print a warning (or an info message) explaining what happened.

That way we keep the current behavior while preserving the stability of manifests. In the same time, we nudged people towards explicitly using carets.

[gordonklaus]

I haven't heard this argument mentioned yet:

• Go prefers explicitness. Dep, by extension, should prefer explicitness. Thus, bare versions should not be allowed.

@ibrasho, your suggestion doesn't sound silly to me.