Quicken, but not specialize, on first execution.

[markshannon]

Currently we hold a per-code-object count to determine when to quicken, and then specialize on the first execution after that.
Instead we could quicken on the first execution, specializing some number of executions after that. Quickening would be done per-instruction, as we do for specialization.

To do this we would insert a QUICKEN instruction in place of any adaptive instructions.
So, for instruction XXX with adaptive form XXX_ADAPTIVE, we currently have:

XXX oparg
0 # Counter

On quickening we change this to:

XXX_ADAPTIVE oparg
0 # Counter

We could instead emit this sequence:

QUICKEN oparg
XXX 0

The QUICKEN opcode would replace itself with the adaptive form of following opcode, and set the following instruction to the warmup value:

XXX_ADAPTIVE oparg
initial_counter # Counter

This would remove the need for the quickening forms of backward branches and RESUME, at the cost of the new instruction QUICKEN.

[gvanrossum]

I am a little vague on the exact definitions of quickening vs. specializing. Is this correct?

• Quickening replaces a "basic" opcode (e.g. LOAD_GLOBAL) with an "adaptive" opcode (e.g. LOAD_GLOBAL_ADAPTIVE).
• Specializing replaces the adaptive opcode with a specialized opcode (e.g. LOAD_GLOBAL_MODULE or LOAD_GLOBAL_BUILTIN).

(UPDATE: PEP 659 doesn't use the terms in exactly this way -- it seems to agree on the definition of specialization, but the entire process seems to be referred to as quickening: "Quickening is the process of replacing slow instructions with faster variants.")

Regarding this idea in particular, it looks like all these instruction sequences take up the same amount of space, so why bother with the unquickened form? If you want to avoid exposing the adaptive opcodes to Python code via the co_code attribute, why not implement the mapping in the co_code getter function (property)?

[markshannon]

I am a little vague on the exact definitions of quickening vs. specializing.

You're not the only one 🙂 I should fix up PEP 659 to be more precise.
Your definitions are as good as any, and what I mean when I'm being careful.

We need to convert from "unquickened" to "quickened" form somewhere.
We have four options:

1. Do it in the compiler, all code is quickened, there is no unquickened form.
2. Do it when unmarshaling.
3. Use a per-code-object counter. This is what we do now
4. Embed the quickening into the bytecode using a QUICKENING instruction

There are downsides to 1-3.

1. We want initial counter values to be non-zero, so the pyc format will depend on endianness. This is probably a fatal flaw for an otherwise appealing option.
2. This means that marshal needs to understand internals of the code objects and makes freezing a pain.
3. Requires the VM to track a use count, requires special purpose code in several places, and may quicken cold bytecode.

#338 proposes a different QUICKENING instruction that also unpacks the bytecode.
I think #338 is the better option long term, but probably isn't feasible for 3.11.
This approach is feasible for 3.11.

[gvanrossum]

@markshannon

2. Do it when unmarshaling.

This one always strikes me as odd, since unmarshaling isn't the only source of code objects. It would seem this always has to be combined with one of the others (I'd imagine (1)).

@ericsnowcurrently

Does anyone treat .pyc files as portable? Isn't the pyc header already beholden to endianness, due to the magic number?

Hmm. It is my understanding that in fact they are portable. At least that's how I understand the header-reading code in importlib/_bootstrap_external.py. E.g. for the magic number:

MAGIC_NUMBER = (3488).to_bytes(2, 'little') + b'\r\n'

This would seem to indicate it's always little-endian. And the opcode format is (currently) always the same as well (one byte opcode, one byte oparg).

I'm guessing to keep .pyc files portable (which I think we should not break!) we may have to use one-byte counters?

[ericsnowcurrently]

Yeah, I misread that magic number code.

[JelleZijlstra]

I believe there are some projects that rely on distributing pyc files.

Can't we make the counter little-endian regardless of the OS's endianness? I suppose that might be inefficient though.

[gvanrossum]

A solution might be to bias the counters so that the default can be zero in the marshaled version.

[brandtbucher]

Or, set the initial value to 0xFFFF and use right shifts instead of subtractions.