Rust support across macOS and Windows versions

[XAMPPRocky]

Currently we support the following targets for Windows & macOS. An issue that has crept more over time is that the specific OS versions that were initially put for support, are no longer tested and built by Rust. This is for a variety of reasons, the main one being that we are bound by the OS versions provided by our CI providers.

platform	OS
i686-apple-darwin	32-bit OSX (10.7+, Lion+)
i686-pc-windows-gnu	32-bit MinGW (Windows 7+)
i686-pc-windows-msvc	32-bit MSVC (Windows 7+)
x86_64-apple-darwin	64-bit OSX (10.7+, Lion+)
x86_64-pc-windows-gnu	64-bit MinGW (Windows 7+)
x86_64-pc-windows-msvc	64-bit MSVC (Windows 7+)

Potential Solutions

• Update Rust's target guarentees to the latest releases.
• Update our platform targets to be OS aware.
  • Not sure how effective this would be as currently Apple and Microsoft have two completely different release schemes. Apple annually releases OS's with breaking changes, and Microsoft will likely not publish a new numbered version of Windows any time in the near future.

[cuviper]

• Update Rust's target guarantees to the latest releases.

Or at least limit ourselves to vendor-supported releases, although that doesn't solve the CI issue.

[XAMPPRocky]

Update Rust's target guarentees to the latest releases.

One thing I forgot to include about this point is this is technically a breaking change. We did break it for 32-bit because there is no vendor or CI support. Upgrading the latest releases for these platforms would be more of an issue as there are not current versions that do have vendor support and significant users. Namely the previous two Apple releases, and Windows 7.

[pietroalbini]

I don't have an opinion on which versions we want to support, but (with my release hat) I strongly think we should commit to something, and actively test that we're still supporting it.

It can be "We test Windows 7 onwards", or "We test the latest 3 macOS releases", or a different policy for each target, but we need to commit to it.

[XAMPPRocky]

@pietroalbini I agree with that, however given the CI situation (at least as I understand it) we'd be more likely to say we support Windows 10 and onwards, which might be a more controversial change.

[pietroalbini]

I agree with that, however given the CI situation (at least as I understand it) we'd be more likely to say we support Windows 10 and onwards, which might be a more controversial change.

But that's what we're guaranteeing in practice today. Yeah, we're probably going to fix a Windows 7 bug if people report it, but we still don't run the tests on that platform ourselves (which is a requirement for being tier 1, both in the current understanding and in the RFC draft).

[cuviper]

One thing I forgot to include about this point is this is technically a breaking change.

I think we need a better definition of breaking change here, because it's not reasonable to expect that we'll continue supporting a given OS forever, especially when its vendor doesn't.

[XAMPPRocky]

I agree. I do think there's merit in supporting an older platform if there's a significant enough community built around it, even if the vendor chooses not to support it. I don't think that's the case here though.

[Centril]

So I think that the terminology "breaking change" here wrt. targets is quite different than the lang/libs sense of RFC 1122 and API evolution. It's fundamentally less permanent and more best effort (e.g. based on CI, vendor support, time, etc.) due to the issues @cuviper brings up.

I also think that the support policy can be per-target or per-OS like what @pietroalbini suggested. That seems quite sensible due to differing levels of support from each OS/platform.

(That said, I don't have specific expertise wrt. macOS to offer, so I'll keep my comments to general policy.)

[Mark-Simulacrum]

I don't have an opinion on which versions we want to support, but (with my release hat) I strongly think we should commit to something, and actively test that we're still supporting it.

But that's what we're guaranteeing in practice today. Yeah, we're probably going to fix a Windows 7 bug if people report it, but we still don't run the tests on that platform ourselves (which is a requirement for being tier 1, both in the current understanding and in the RFC draft).

I don't personally see that it is necessarily true that what we guarantee and what we test need to coincide. That seems like an extremely high bar, too high for our purposes! I think we should document what we currently test on CI, and what we intend to support beyond that (i.e., where we will dedicate developer time, prioritize bugs, etc.). I think that making it mandatory that anything we claim to support is fully tested on CI is too high a bar; it would unduly limit the support structure we can give. I suppose in some sense this means that only what we test on CI is a Tier 1 platform per I believe current definitions (tests + builds available) whereas previous platforms are Tier 2 (builds available, tests not run).

For example, with macOS, it is my impression that testing anything except the latest release is not really super tenable with our current and future CI providers. But I would expect us to still support (say) the last three releases of macOS; we would not test that support explicitly, but any reported bugs would be prioritized as if it was the latest release of macOS. I don't know how far back we would go (maybe three releases, like what Apple does it seems).

[BatmanAoD]

@Mark-Simulacrum This seems to point to a longer discussion we should perhaps have. Our tier guarantees are defined primarily in terms of CI, so it seems to me that there may be some value in brainstorming a way to maintain some kind of relationship there. One idea I just had is rotating which version of a given platform we test with for every test cycle, so that we're still using the same amount of compute resources to do a single test run, but over time we'd catch regressions on specific versions.

[Mark-Simulacrum]

AIUI, it's not a resourcing problem -- we literally don't have CI providers that support macOS 10.13, for example.

[pietroalbini]

Hmm, ok, given the discussion I'd be fine with not testing everything on CI. With my release hat I'm not happy about it, but with my infra hat I don't know how to actually test on every platform without setting up custom CI infra.

[XAMPPRocky]

Okay, how do we communicate this change? Is this something that needs to be part of the Target Tier Policy RFC, its own RFC, or just an update to the forge documentation?

[Mark-Simulacrum]

To be clear, I believe that the change we are considering is changing the minimum OS X version to 10.13 (from 10.7), which falls in line with Apple support (per rust-lang/rust#50776 (comment)).

I believe that we should prepare an RFC, similar to @pietroalbini's 32-bit Apple demotion to tier 3 RFC, and aim to land that. The RFC would be slightly different in that it would set a minimum bar that on Apple, we support no earlier than latest publicly security-release targeted versions by the vendor (only Apple, in this case). As of today, that's 10.13.