MSRV policy for libc crate

[joshtriplett]

In a previous libs team meeting, we discussed MSRV policy for the libc crate, and we debated between an N-1 policy and a slightly more conservative policy such as N-3 or N-4. We ended the meeting without having settled on a specific consensus, though it seemed like we were roughly on the same page.

Rather than make ad-hoc changes based on a non-specific MSRV policy, I think we should go ahead and nail down an MSRV policy for present and future changes.

I'd like to propose the following concrete policy:

• Support the last three stable versions of rustc (stable and the prior two versions).
• Only bump MSRV when we use a feature requiring newer Rust. (We do not systematically bump MSRV each time a stable release occurs.)
• Treat MSRV increases as an ordinary change, not a breaking change.

Note that this is a policy proposal for libc, not for other crates maintained by the libs team. After we get consensus on an MSRV policy for libc, we can consider what policy we want for other crates. Nonetheless, libc MSRV policy seems likely to serve as a defacto lower-bound for supported Rust versions among other libs team crates, and among many crates in the ecosystem.

For further information on the benefits of this: the PR dropping support and version detection for older versions of rustc was able to remove ~1600 lines of code, and that's not counting subsequent work that may be able to drop some of the wrapper macros. This represents a substantial improvement to maintainability, and may make it easier in the future to move to more automation to keep libc updated with the latest platform APIs.

[joshtriplett]

@rfcbot merge

[rfcbot]

Team member @joshtriplett has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Amanieu
• @BurntSushi
• @dtolnay
• @joshtriplett
• @m-ou-se
• @yaahc

Concerns:

• opportunity for ecosystem to give feedback (MSRV policy for libc crate #72 (comment))

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[petrochenkov]

This is such an unreasonably short support window for a foundational crate like libc.
I would expect at least couple of years of supported versions instead of N-3,4

[thomcc]

Yeah it's a bit tough, since this impacts a lot of crates that have libc in their dep tree. I think being conservative here is worthwhile, unfortunately.

One danger here is if a lot of crates end up pinning libc versions (which they shouldn't¹, but in practice is the tool many people reach for), those crates become incompatible to use with eachother.

Footnotes

1. Allowing users with a lockfile to deal with it via cargo update --precise is better. ↩

[joshtriplett]

@petrochenkov Assume, for the moment, that we aren't looking to support the use case of "run brand new crates from crates.io on a multi-year-old snapshot of rustc/cargo in an enterprise distribution". Are there any other use cases for which "couple of years" is actually a specific end-user need? Not something where crate-author asks crate-author asks crate-author based on hypotheticals, but actual end users who have an old rustc, and a need for new crates, and no better options?

[BurntSushi]

My instinctual response to this is also, frankly, that N-4 is too aggressive. But:

Are there any other use cases for which "couple of years" is actually a specific end-user need? Not something where crate-author asks crate-author asks crate-author based on hypotheticals, but actual end users who have an old rustc, and a need for new crates, and no better options?

I basically don't have a great concrete answer to this. And I will say, I've personally been looking for one for a while. Any time someone tells me about wanting crates to compile with older versions of Rust, I always try to ask: why? From memory, I've basically gotten these answers:

1. "I did sudo apt-get install rustc cargo, cloned your repository and then tried to run cargo build." In effect, they are trying to port workflows that they expect to work (probably from other language ecosystems) to Rust, and then get miffed that compilation breaks. Sometimes this even translates into errant bug reports. (Although, I think that's largely solved by Cargo's rust-version feature, once I get around to using it.) I also think people interpret this as a signal that Rust is "too unstable," in the sense that too many people are using language features that require a very new compiler. My experience is that most C or C++ libraries don't do this, and tend to update their language version (e.g., C++11 to C++14) pretty conservatively. So even folks using a conservative Linux distro will be able to do the apt-get install foo and then build random projects using foo. But for Rust, that doesn't really work. Instead, they really have to go use rustup, which is just going to be foreign to lots of folks not familiar with the Rust ecosystem.
2. "My company pins its Rust version in CI and it takes a lot of resources to upgrade." I've only heard this one once or twice, and I didn't really get a ton of details out of it. The obvious question to ask here, then, is whether they in turn need to use crates that require a new Rust. My recollection (I couldn't find a record of the discussion) is that the Rust ecosystem moved so fast relative to how often they upgraded their Rust compiler that it was not uncommon for them to try to bring in a crate and then have it fail to compile on their version of Rust. I don't recall drilling into this any more, e.g., why they couldn't "just" use old versions of crates. (Which does seem pretty annoying to me.) I think the overall impression I got was that this just wasn't typically a problem for them when using C or C++.

I think (1) is basically some combination of an awareness/workflow/perception problem. I think there's probably something to (2), but I just don't know enough concrete details about any specific use case to really know for sure. And in particular, I suspect (2) is kind of also a perception problem. Both (1) and (2) seem somewhat rooted in the idea that Rust and its ecosystem move so much faster than the things it tends to supplant that it tends to be really noticeable.

Now I do think perception problems are real problems, but I'm not convinced they reflect concrete use cases that argue strongly for a conservative MSRV policy. (By "conservative," I'm thinking that there is roughly support for N-9 releases, i.e., one year.)

Note that this is a policy proposal for libc, not for other crates maintained by the libs team. After we get consensus on an MSRV policy for libc, we can consider what policy we want for other crates.

OK, so I think this is really where my real concern with this change is. libc is basically as foundational as you can get as a library. Deciding an MSRV policy for it comes very close to effectively deciding an MSRV for the broader ecosystem. IMO, even if the MSRV is really just a perception problem, that's a pretty big deal. (I say "very close," because as @thomcc mentioned, if people hate this enough, they'll starting pinning libc and then that's going to be a cure that is worse than the disease. So technically, users of libc can still enforce their MSRV by either pinning libc in libraries (very bad) or pinning libc in their Cargo.lock (which is totally fine). But other than that, libc will end up deciding the MSRV for a lot of folks I think.)

Since libc is foundational and because I suspect this policy is probably more aggressive than extant MSRV policies in the ecosystem (being a little hand wavy there), I do think that we should try to broadcast this policy question before making a final decision on it. That is, I would like to make a good faith effort to give folks an opportunity to provide their opinion on this because of the impact this will have on the broader ecosystem. Basically, I want to avoid a scenario where this happens:

• We do an FCP here and because relatively few people watch this repo, it passes silently.
• The libc MSRV policy is put into place.
• Six months from now, we finally have a need to "test" the policy because there's this new Rust feature or API that was introduced in Rust N-4. So that gets merged and shipped out.
• A bunch of people who care about MSRV realize that the new release of libc has almost nearly forced them to make their policy more aggressive than what they had end up asking what's going on in the libc tracker.
• They get pointed to this issue and wonder why such a change wasn't broadcasted more widely so that they could have the opportunity to have their voice heard.

Maybe I'm overplaying it, but that's my sense. So I'll register a concern for that:

@rfcbot concern opportunity for ecosystem to give feedback

Otherwise, for me personally, I don't think I feel strongly for or against this change. I've long wanted good reasons for a conservative MSRV policy, and I've had a hard time finding them. Maybe this issue will force out some good ones that I've not heard of before. But I can definitely empathize with the annoyance of having to wait so long to use new Rust stuff, so I get where this is coming from.

[joshtriplett]

Note that this is a policy proposal for libc, not for other crates maintained by the libs team. After we get consensus on an MSRV policy for libc, we can consider what policy we want for other crates.

OK, so I think this is really where my real concern with this change is. libc is basically as foundational as you can get as a library. Deciding an MSRV policy for it comes very close to effectively deciding an MSRV for the broader ecosystem.

I was attempting to narrow the question to libc, not because I think it likely that other official crates can or should realistically be more conservative than libc, but rather, I wanted to separate the questions so that if someone wanted to argue "we should support X" they didn't have to simultaneously argue whether that meant X for libc or for everything.

If we do agree on N-2 for libc, then our realistic choices of policies for other official crates are N-2, N-1, or N. I'd tend to suggest N-2 for "foundational" crates like libc or libz-sys.

[petrochenkov]

2. "My company pins its Rust version in CI and it takes a lot of resources to upgrade."

I more or less meant this scenario.
In my case it was more like "several teams need to build the code in several infras, and asking others to update their infra too often is a nuisance because they have other more important things to do, so in practice the required Rust toolchain gets updated once in 0.5-1.5 years".

Crate versions still can be updated more often because it doesn't require similar coordination.
Also few crates change MSRV that often in practice so most of the dependencies still build in a year and don't need to be pinned to prevent updates. So the N-3,4 policy actually makes libc move much faster than majority of other less fundamental crates.

[shepmaster]

So the N-3,4 policy actually makes libc move much faster than majority of other less fundamental crates.

Not necessarily, but only possibly:

Only upgrade if there's a feature we actually need.

It just means that libc could bump the MSRV if needed. As you said...

few crates change MSRV that often in practice

[BurntSushi]

@petrochenkov Can you say more please? Here are the things I'm personally interested in:

• Why do you think it's justified for the (probably the entire) Rust project to spend effort maintaining a conservative MSRV because it is costly for y'all to upgrade? To me, it feels like this hinges a lot on just how common your scenario is. Like, what scale of problem is this? Is this a "I'm a little annoyed" flavor of problem, or is this a "we'll probably have to stop using Rust" showstopping problem?
• If upgrading Rust is really that costly, can you say more about why using older versions of crates isn't an acceptable alternative path for you?

@shepmaster

So the N-3,4 policy actually makes libc move much faster than majority of other less fundamental crates.

Not necessarily, but only possibly:

Only upgrade if there's a feature we actually need.

It just means that libc could bump the MSRV if needed. As you said...

few crates change MSRV that often in practice

I think you're technically correct, but I think the color you're adding here might give the wrong impression. There's a burden to supporting older MSRVs, and while @joshtriplett's post does include the language "Only upgrade if there's a feature we actually need," I think the word "need" there is doing a lot of work. For example, "need" might mean, "we want to use a new std API and don't feel like writing version sniffing code in build.rs to do conditional compilation" or it might mean "we have so much version sniffing already that adding more is just not worth the burden" or it might mean "we want to use a new Cargo.toml feature and there's no easy way of introducing such things conditionally." Frankly, all three are somewhat reasonable interpretations, but it's not totally clear which one will be followed in practice to me. So it might worth clarifying what exactly is meant by "need" here.

For example, if "need" is interpreted broadly, then I think it's very likely that libc will end up bumping MSRV more rapidly than existing "core" crates. But if "need" is interpreted narrowly, then it might actually be pretty rare for libc to bump MSRV.

[BurntSushi]

Someone else I'd like to hear from is @dtolnay. As I understand it, you maintain a Rust 1.13 MSRV for serde. This is despite requiring at least some non-trivial effort to do so. I don't think I've seen you opine on why exactly you do this. Could you share your perspective here?

[BurntSushi]

(Also, I have not actually broadcast this issue to a wider audience yet. Are people okay if I do that? Or do folks think we should wait on that for a bit or go about it a different way? As a former mod, I always feel a little weird shining spotlights on GitHub issue threads, since it can go sideways pretty easily.)

[petrochenkov]

@BurntSushi

Why do you think it's justified for the (probably the entire) Rust project to spend effort maintaining a conservative MSRV because it is costly for y'all to upgrade?

Because it's not that much effort in practice.
It's "some extra maintenance in libc alone" vs "some extra maintenance for all the people with slower Rust toolchain update schedules", the choice seems pretty clear to me.

If upgrading Rust is really that costly, can you say more about why using older versions of crates isn't an acceptable alternative path for you?

It's acceptable, and the issue is not a showstopper, but newer crate versions are not purely cosmetic and do fix relevant bugs sometimes (or add relevant Linux constants or whatever, in case of libc).