Improvements to the dispatching logic in the interpreter

[markshannon]

No big ideas here. Just cleaning up the code around dispatching so it does less work and thus runs faster.

I don't expect to see much, if any, speed up on Linux, but Windows should gain parity (approximately) by allowing the compiler to combine the jumps.

1. Move tracing logic out of fast dispatch path:

if (tracing_possible) {
    goto tracing_dispatch;
}
fetch_and_decode();
#ifdef COMPUTED
     goto *dispatch_table[opcode]
#else
     goto dispatch_opcode
#endif

2. Remove anything between the label and the switch:

dispatch_opcode:
    switch(oparg) {

Which should allow the compiler to combine the goto and the computed jump.

We may need to generate cases for all undefined opcodes, so that the range check is guaranteed to be eliminated.

[gvanrossum]

Doesn't the presence of a 'default' case eliminate the range check?

[markshannon]

Probably not. Adding a single case for 255 might do the job, as all the default values would then be within the range 0-254.

[markshannon]

MSVC seems quite picky about when it will merge the goto and computed branch for the switch into a single branch, but it can be persuaded to do so (at least on this online compiler):

https://godbolt.org/z/nss93dr7T

It seems that MSVC is quite keen on merging similar code paths.
If the code immediately before the dispatch: label resembles DISPATCH to closely, we get the same code as if a default we used.
If all the illegal opcode cases are written as oparg = opcode; instead of oparg = LITERAL_VALUE then the cases are merged and we get jump merging, but with an extra memory read. This might be an acceptable compromise as it should still help the branch predictor.

I don't think we can directly test that the code is compiling as we wish.
We could check it manually initially and then rely on performance testing to show up any regressions.
For that we would need a Windows machine for performance testing.

[gvanrossum]

FWIW I am using a Windows machine with VC 2017 loaded on it as my development machine. I can probably find out how to inspect the assembly code quite easily, if needed.

[markshannon]

faster-cpython/cpython#3

[markshannon]

I'm breaking this into three tasks.
The first two can be done now. The last will have to wait until we know if it works.

• Minimize the overhead of checking if tracing is turned on by moving it to a C local variable.
• Modify DISPATCH() to minimize the difference between switch and computed-goto forms.
• Add code for all unknown opcodes to persuade MSVC to use computed jumps inline (once we know that it works).

[markshannon]

Splitting the third part off into its own issue and closing this.