all: add GOAMD64 environment variable

[mdempsky]

This proposal is to add a GOAMD64 environment variable, with the initial options of "baseline" (default), "v2", and "v3".

Most Go architectures support a GO[arch] environment variable to control architecture-specific options: GO386, GOARM, GOMIPS, GOMIPS64, GOPPC64, GOWASM. However, the AMD64 port (presumably the most common architecture Go is deployed on) still limits itself to the original, now-20-year-old instruction set, with some occasional runtime CPUID detection when the savings is significant enough to merit it. (For comparison, GOPPC64 supports optimizing for power9, which only became available in 2017.)

This is further complicated by x86-64 having accumulated many, many instruction set extensions, with each processor revision having a different set of supported extensions. Making users responsible for deciding what set of extensions to enable doesn't feel very Go-like.

However, in 2020, the x86-64 psABI added four named microarchitecture levels to help group the extensions: "x86-64 (baseline)", "x86-64-v2", "x86-64-v3", and "x86-64-v4". See https://en.wikipedia.org/wiki/X86-64#Microarchitecture_levels or https://developers.redhat.com/blog/2021/01/05/building-red-hat-enterprise-linux-9-for-the-x86-64-v2-microarchitecture-level/ for further details.

The "baseline" corresponds to what Go already supports, while "v2" and "v3" each add some new instructions that could be useful for Go programs (e.g., POPCNT in v2, BMI1/BMI2 in v3).

v2 CPUs appear commonplace today. E.g., RHEL9 will only support v2, per the above blog post; all GCE CPUs support v2, and I believe all AWS and Azure CPUs too.

v3 CPUs are also increasingly common. E.g., only GCE's Ivy Bridge and Sandy Bridge CPUs are limited to v2; Haswell (launched 2013) and newer support v3.

On issue #25489, I reported results from two optimization attempts at using Haswell's BMI instructions (PEXT for varint decoding, LZCNT and a couple others for scanobject). These are optimizations that could benefit from targeting v3 CPUs specifically, but probably wouldn't be worthwhile if they needed to rely on runtime CPUID detection.

It's also been suggested that at process startup, the Go runtime should throw if it's been compiled to assume instruction set extensions that aren't available on the CPU. I think that's a good idea.

Questions:

• Are "baseline", "v2", and "v3" the best names? "v1" would perhaps be better than "baseline", but the psABI doesn't formally name it that. We could suggest that though?

• Should we add "v4" too? This only adds AVX512 instructions, which the Go compiler/runtime don't immediately have any use for, and which seem a bit contentious about whether to use them on current processors anyway.

[ianlancetaylor]

Since v4 is defined I would be inclined to say that we should accept v4 as a valid GOAMD64 value but treat it as v3.

[mdempsky]

Since v4 is defined I would be inclined to say that we should accept v4 as a valid GOAMD64 value but treat it as v3.

I think it's okay if we accept GOAMD64=v4 and don't actually use any of the AVX512 instructions. But then I think the runtime should probably still check that they're available at runtime, so if we decide to start using AVX512 in the future we won't have to worry about users erroneously running v4 binaries on v3 CPUs.

[martisch]

I would like to note that the Pentium and Celerons (often used in low tier laptops, NUCs and NAS devices) do not support AVX/AVX2 and while based on Haswell and newer (and are categorized with the same architecture names) are v2 and not v3. So for server farms where performance matters v3 is likely a good choice but for general computing even on newer chips v2 is still relevant.

[mvdan]

I think it's also worth noting that some mainstream Linux distros are looking at adopting the same microarchitecture levels for their binary packages. For example, Arch will add v3 to their mirrors on top of the existing "baseline": https://gitlab.archlinux.org/archlinux/rfcs/-/merge_requests/2/diffs

Assuming they ship this soon, I imagine any packages building with GCC or LLVM would benefit, and Go packages would be left behind without this proposal.

[rsc]

It's nice to see Intel and AMD coalescing on fewer configuration knobs.

[rsc]

"baseline" is an unfortunate name because it sounds like "Go's default".
(Compare with the mentions of baseline in discussions of GOEXPERIMENT.)
It does seem like "v1" is the obvious choice for the base configuration.
Maybe someone can suggest that to Intel?

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[beoran]

While I agree that adding support for different API levels of the Go AMD64 port, I would like v1 to stay the default for Go applications and for the Go compiler itself at least for the next 10 years. My family and I use old refurbished computers with Linux, since that still works fine, and I think there must be many others around the world who are in the situation of not having access to recent hardware.

[rsc]

@beoran, what the default or minimum requirements are for Go would be different proposals. As I understand it, no one is proposing to change the default or the minimum requirement away from v1 in this issue. This is just about adding an architecture setting similar to GOARM and others.

[rsc]

If we can call the current baseline "v1" instead of "baseline" then it seems like everyone is on board.
Do I have that right?