Configure Tier 2 interpreter at build time

[gvanrossum]

It would be nice if by default we didn't ship the optimizer infrastructure, since approximately nobody is going to use it, so we should ship (and by default build) binaries that just don't include it.

I propose to enable it only when configured with --enable-experimental-jit-interpreter. We drop support for PYTHON_UOPS and -Xuops, in favor of PYTHON_JIT=0|1.

@mdboom this will have some consequences for the benchmarking infra; checking the Tier 2 box should use this configure option rather than setting the PYTHON_UOPS=1 env var. It should be possible to use both (./configure gives a warning but otherwise seems to ignore incorrect flags, and the env var will just be ignored).

Linked PRs

• gh-118335: Configure Tier 2 interpreter at build time #118339
• gh-118335: Make REGEN_JIT_COMMAND empty if tier2 interpreter enabled #118493
• gh-118335: Rename --experimental-interpreter on Windows to --experimental-jit-interpreter #118497

[mdboom]

@mdboom this will have some consequences for the benchmarking infra; checking the Tier 2 box should use this configure option rather than setting the PYTHON_UOPS=1 env var. It should be possible to use both (./configure gives a warning but otherwise seems to ignore incorrect flags, and the env var will just be ignored).

That should be easy enough to deal with, but I'll hold off on shipping that change until #118339 is merged.

[gvanrossum]

That should be easy enough to deal with, but I'll hold off on shipping that change until #118339 is merged.

Understood, but then how can I run a benchmark that tells me how Tier 2 fares with my changes before merging it? (Possibly we won't care. We should discuss this in today's meeting.)

[mdboom]

Oh, good point. If I'm reading correctly, we could probably do both the ./configure flag and the environment variable for a while (and then remove the environment variable).

[gvanrossum]

Off-line we decided on a different configure experience. The --configure-experimental-jit flag will be extended so you can write

./configure --enable-experimental-jit={off|yes|no|interp}

(enum names subject to bikeshedding) which selects whether the JIT or the Tier 2 interpreter is built, and whether it's on or off by default. An environment variable, PYTHON_JIT={0|1} can override whatever is built.

[gvanrossum]

Meanwhile, Brandt and I solved the Windows failure, which was due to _Py_JIT being undefined when _testinternalcapi.c was being compiled in JIT mode. Fix forthcoming.