Faster builtin functions by performing more work at specialization time.

[markshannon]

A lot of the work done by builtin functions, especially simpler ones, is argument parsing, type checking, unboxing and boxing.

If you take a look at PyArg_ParseTupleAndKeywords you will the see the amount of work that needs to done to handle the general case.

Many builtin functions have five phases:

1. Parse arguments
2. Type check arguments
3. Unbox arguments
4. Do the actual work
5. Box any result

Argument clinic already generates wrappers that break the above down into 3 phases:

1. Parse, type check and unbox using PyArg_Parse...
2. Do the actual work
3. Box any result

We can change this so that the argument clinic generated code skips the parse phase:

1. Type check arguments
2. Unbox arguments
3. Do the actual work
4. Box any result

The interpreter can do the "0th" phase of parsing the arguments.

This would work as follows:
Add a new METH_N calling convention that takes exactly N arguments, as defined in the MethodDef struct.
So, if N were 1, the function pointer would have the signature f(PyObject *callable, PyObject *args[1])
There would need to be an upper limit on N, probably about 6.

The vectorcall implementation of this would need to do the parsing, but that's no less efficient that what we do at the moment.

For example consider the builtin method str.encode with the signature encode(self, /, encoding='utf-8', errors='strict').
If we call "hi".encode(errors="ignore") there is a lot of parsing of arguments that has to be done for every call.

With METH_N we can have argument clinic define this function:

PyObject *str_encode(PyObject * _unused, PyObject *args[3])
{
     PyObject *self = args[0];
     PyObject *encoding = args[1];
     if (encoding == NULL) {
          encoding = &PyGlobals.strings.utf_8
     }
     PyObject *errors = args[2];
     if (errors == NULL) {
          errors = &PyGlobals.strings.strict
     }
     return str_encode_impl(self, encoding, errors);
}

Which is reasonably slick.

Note that we must call str_encode with exactly 3 arguments, correctly parsed. This is fiddly in the vectorcall wrapper, but can be specialized nicely.

Going back to our example of "hi".encode(errors="ignore") the three arguments we should be passing are "hi", NULL, "ignore"
We can parse the arguments at specializing time, creating a permutation array that can be evaluated quickly for each call.
One way that could work is:

1. Unconditional push NULL to the stack.
2. Do a series of swaps to get the stack in the correct layout.

Most calls don't do anything fancy, so the we would probably special case the "already in the right order" case.

Implementing this

This only really works if we can move most existing builtin functions to the new form.
To do that we need to make it:

1. Easy to use, the implementation function should be the same as, or simpler than, the current argument clinic form.
2. It should be faster than the current form in all cases when called from Python code.
3. It shouldn't be slower than the current form when called using the vectorcall calling convention.

Some open questions

There are a few details I've glossed over in the above discussion:

1. Where do we store the signature? We need the signature for specialization and vectorcall. The signature could be represented as something like a _PyArg_Parser struct.
2. How do we check for and handle errors passing the wrong number of arguments? We could NULL terminate the array, at least for debug builds, but would that be sufficient?

[gvanrossum]

Isn't there a near-infinite number of builtin functions and methods that we'd have to convert this way before it pays off? Even if we could collect the set of most-used builtins from PyPerformance (or the Bloomberg demo for that matter), that wouldn't necessarily translate to other apps.

[markshannon]

It's up to the extension authors. If they want their extensions to be fast, they can use this. I don't expect any immediate results.

[cfbolz]

One possible idea in this general space would be to make cython-generated extensions use the new mechanism automatically, which would bring the benefits to a whole bunch of extensions at once.

[markshannon]

It looks like we might get results for this sooner rather than later.

The latest stats show large slowdowns for the regex benchmarks relative to 3.10, which is (from a fairly superficial inspection) due to non-specialization (and thus repeatedly cycling though the specializer) of re.Pattern.search and maybe other methods.
In this particular case, re.Pattern.search uses the flags METH_METHOD|METH_FASTCALL|METH_KEYWORDS

Replacing all these special cases with METH_N would be profitable.

[markshannon]

@cfbolz Would this be useful for PyPy?