How is type of cls and self inferred as type[Self] and Self?

[alwaysmpe]

I've created a PR python/typeshed#12952 that gives functools.cache and functools.lru_cache correct type signatures. As part of that I've been messing with the descriptor protocol a fair bit and I think it's possible that PR could be simplified. If you consider the below:

from __future__ import annotations
from collections.abc import Callable
from typing import Concatenate, Self, overload, assert_type, Protocol

class MyClassmethod[T, **P, R]:
    def __init__(
            self,
            fn: Callable[Concatenate[type[T], P], R]
    ) -> None:
        ...
    @overload
    def __get__(self, instance: None, owner: type[T] | None = None, /) -> Callable[P, R]: ...
    @overload
    def __get__(self, instance: T, owner: type[T], /) -> Callable[P, R]: ...

class MyAltClassmethod[T, **P, R]:
    def __init__(
            self,
            fn: Callable[Concatenate[T, P], R]
    ) -> None:
        pass
    @overload
    def __get__(self, instance: None, owner: type[T] | None = None, /) -> Callable[P, R]: ...
    @overload
    def __get__(self, instance: T, owner: type[T], /) -> Callable[P, R]: ...


class MyProperty[T, **P, R]:

    def __init__(
            self,
            fget: Callable[Concatenate[T, P], R],
    ) -> None:
        ...
    
    @overload
    def __get__(self, instance: None, owner: type[T] | None = None, /) -> Self: ...
    @overload
    def __get__(self, instance: T, owner: type[T], /) -> R: ...

class Cache[**P, R](Protocol):
    def __init__(self, fn: Callable[P, R]):
        ...
    def __call__(self, *args: P.args, **kwargs: P.kwargs) -> R:
        ...
    def cache_clear(self) -> None: ...

def cache[**P, R](fn: Callable[P, R]) -> Cache[P, R]:
    ...

class MyCls:

    @MyProperty
    @cache
    def foo(slef) -> int:
        ...

    # type error - cls is of type Self,
    # type[Self] expected
    @MyClassmethod
    @cache
    def cls_mtd(cls, val: int) -> int:
        ...

    @MyAltClassmethod
    @cache
    def alt_cls_mtd(cls, val: int) -> int:
        ...
    @classmethod
    @cache
    def bi_cls_mtd(cls, val: int) -> int:
        ...

assert_type(MyCls().foo, int)
assert_type(MyCls.alt_cls_mtd(1), int)
assert_type(MyCls.cls_mtd(1), int)
# type error - cls argument missing 
assert_type(MyCls.bi_cls_mtd(val=1), int)

The only version of classmethod that doesn't cause a type error is MyAltClassmethod, and that's implementing the classmethod descriptor protocol without the constraint that the first argument is a type. But if the typeshed property and classmethod typeshed decorators were modified to correctly implement the descriptor protocol, the PR I've created would get simpler. So I found myself asking, is the cls argument being type[Self] just a special case for when classmethod is present, but only after that descriptor has been applied? And likewise, the property definition in typeshed just returns Any so I'm guessing there's a special case in pyright and if that were fixed too it'd be better.

A lot of the complexity in my PR is because when applied to a descriptor, classmethod seems to stop doing what it's supposed to do and there's no easy way to describe nesting of descriptors.

[erictraut]

classmethod and staticmethod require significant special-casing in a type checker. They are unlike any other decorators. Most decorators act like regular function calls, and a type checker treats them as such. For regular function calls, the types of the arguments are evaluated first. Then the type checker validates that the arguments and their types are compatible with the call's signature, possibly solving any type variables based on the constraints imposed by those argument types. This approach doesn't work for classmethod and staticmethod because their presence affects the type of the argument, which is a callable whose first parameter's type depends on the presence or absence of these special decorators.

The special-casing for classmethod and staticmethod runs very deep in type checkers. These two decorators need to be understood by the earliest phases of analysis — the phase referred to as "binding". This phase occurs before types of symbols can be determined. It's responsible for discovering symbols and determining which scope they belong to — effectively building the symbol table for each scope. The binder needs to know which symbols are accessed through a cls or self object so it can determine which symbols are instance and class variables. That means that the binder in a Python type checker needs to hard-code the symbols classmethod and staticmethod and assume that they will not be aliased or renamed. It also means that you cannot write replacement classes that effectively mimic the implementation details of classmethod and staticmethod. For this reason, the approach you're trying to use above (with MyClassmethod) will not work with pyright. It probably won't work with other type checkers either.