Grand unified tail-calling

[Fidget-Spinner]

Feature or enhancement

Proposal:

We have a tail calling interpreter. I now propose to tightly couple that with the JIT, so we can tail call into the JIT and tail call out of it.

This has a few benefits:

1. No more C stack consumption by the JIT.
2. Much more efficient entry and exit of jitted code. OSR entry is just a single indirect jump.

We need to align the calling convention of the JIT and the tail calling interpreter. This will need careful benchmarking to make sure we don't regress JIT performance.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

• gh-129819: Allow tier2/JIT and tailcall #129820

[Fidget-Spinner]

cc @brandtbucher

[brandtbucher]

The tricky thing here is that it seems like every compiler is going to have their own incompatible ABI for __attribute__((preserve_none)). Meaning, even if GCC and MSVC support it, we're still not able to call directly between them if the JIT is compiled with Clang (which I don't see changing anytime soon).

At worst, though, we just have a little trampoline that moves the arguments around to the expected registers on the way in (depending on the exact ABI, we may need to do more, and do it on the way out, too). But at least it would get rid of the C stack consumption. It's an open question whether that's worth the effort involved to make it work (and keep it working).