Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[mypyc] Optimize dunder methods #17934

Merged
merged 1 commit into from
Oct 18, 2024
Merged

Conversation

jairov4
Copy link
Contributor

@jairov4 jairov4 commented Oct 14, 2024

This change gives mypyc the ability to optionally optimize dunder methods that can guarantee strict adherence to its signature typing. The optimization allows to bypass vtable for dunder methods in certain cases that are applicable.

Currently, mypy has adopted the convention of accept dunder methods that return NotImplemented value even when its signature do not reflect this possibility. With this change and by enabling an special flag, mypyc will expect strict typing be honored and will unleash more optimizations like native call without vtable lookup for some cases on dunder method calls.
For example it could avoid calls to RichCompare Python API making the code can be fully optimized in by the C compiler when some comparison with dunders are required.

Example:

@final
class A:
    def __init__(self, x: i32) -> None:
        self.x: Final = x

    def __lt__(self, other: "A") -> bool:
        return self.x < other.x

A(1) < A(2)

would produce:

char CPyDef_A_____lt__(PyObject *cpy_r_self, PyObject *cpy_r_other) {
    int32_t cpy_r_r0;
    int32_t cpy_r_r1;
    char cpy_r_r2;
    cpy_r_r0 = ((AObject *)cpy_r_self)->_x;
    cpy_r_r1 = ((AObject *)cpy_r_other)->_x;
    cpy_r_r2 = cpy_r_r0 < cpy_r_r1;
    return cpy_r_r2;
}

...
cpy_r_r29 = CPyDef_A_____lt__(cpy_r_r27, cpy_r_r28);
...

Instead of:

PyObject *CPyDef_A_____lt__(PyObject *cpy_r_self, PyObject *cpy_r_other) {
    int32_t cpy_r_r0;
    int32_t cpy_r_r1;
    char cpy_r_r2;
    PyObject *cpy_r_r3;
    cpy_r_r0 = ((AObject *)cpy_r_self)->_x;
    cpy_r_r1 = ((AObject *)cpy_r_other)->_x;
    cpy_r_r2 = cpy_r_r0 < cpy_r_r1;
    cpy_r_r3 = cpy_r_r2 ? Py_True : Py_False;
    CPy_INCREF(cpy_r_r3);
    return cpy_r_r3;
}

...
cpy_r_r29 = PyObject_RichCompare(cpy_r_r27, cpy_r_r28, 0);
...

Default behavior is kept.
Tests run with both of strict typing enabled and disabled.

Should be rebased and reviewed after #17886

@jairov4
Copy link
Contributor Author

jairov4 commented Oct 14, 2024

Hi @JukkaL , can you review this change?

@JukkaL
Copy link
Collaborator

JukkaL commented Oct 15, 2024

I'll try to get to this later this week. This week is pretty busy, since mypy 1.12 release just got out and a bunch of regressions have been reported.

Copy link
Collaborator

@JukkaL JukkaL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, thanks.

I don't want to keep the strict_dunder_typing flag around for long. Maybe it would only be enabled for value types, or we can require always explicitly annotating NotImplemented in return types.

We should have a minimal number of mode flags that impact compilation. This can be sorted out in the future, however.

@JukkaL JukkaL merged commit c9d4c61 into python:master Oct 18, 2024
13 checks passed
@jairov4 jairov4 deleted the mypyc-opt-dunders branch October 19, 2024 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants