functools: singledispatchmethod doesn't work with classmethod

[vr2262]

BPO	39679
Nosy	@ncoghlan, @glyph, @ambv, @mgrandi, @ilevkivskyi, @miss-islington, @tirkarthi, @mental32, @vr2262, @FFY00, @EugenePY, @AlexWaygood
PRs	bpo-39679: Add tests for classmethod/staticmethod singledispatchmethods #29034 [3.10] bpo-39679: Add tests for classmethod/staticmethod singledispatchmethods (GH-29034) #29072 [3.9] bpo-39679: Fix singledispatchmethod classmethod/staticmethod bug #29087

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-10-28.16:03:07.339>
created_at = <Date 2020-02-18.19:16:15.217>
labels = ['type-bug', 'library', '3.9']
title = "functools: singledispatchmethod doesn't work with classmethod"
updated_at = <Date 2021-10-28.16:03:07.338>
user = 'https://github.com/vr2262'

bugs.python.org fields:

activity = <Date 2021-10-28.16:03:07.338>
actor = 'lukasz.langa'
assignee = 'none'
closed = True
closed_date = <Date 2021-10-28.16:03:07.339>
closer = 'lukasz.langa'
components = ['Library (Lib)']
creation = <Date 2020-02-18.19:16:15.217>
creator = 'Viktor Roytman'
dependencies = []
files = []
hgrepos = []
issue_num = 39679
keywords = ['patch']
message_count = 15.0
messages = ['362233', '362253', '362434', '362442', '371230', '372405', '378344', '380221', '395910', '404147', '404348', '404362', '404363', '405195', '405196']
nosy_count = 13.0
nosy_names = ['ncoghlan', 'glyph', 'lukasz.langa', 'markgrandi', 'levkivskyi', 'miss-islington', 'xtreak', 'mental', 'Viktor Roytman', 'FFY00', 'EugenePY', 'dmkulazhenko', 'AlexWaygood']
pr_nums = ['29034', '29072', '29087']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue39679'
versions = ['Python 3.9']

[vr2262]

I couldn't get the example given for the interaction between @singledispatchmethod and @classmethod to work https://docs.python.org/3/library/functools.html?highlight=singledispatch#functools.singledispatchmethod

    from functools import singledispatchmethod
    
    
    class Negator:
        @singledispatchmethod
        @classmethod
        def neg(cls, arg):
            raise NotImplementedError("Cannot negate a")
    
        @neg.register
        @classmethod
        def _(cls, arg: int):
            return -arg
    
        @neg.register
        @classmethod
        def _(cls, arg: bool):
            return not arg
    
    
    if __name__ == "__main__":
        print(Negator.neg(0))
        print(Negator.neg(False))

Leads to

    $ python -m bad_classmethod_as_documented
    Traceback (most recent call last):
      File "/usr/lib/python3.8/runpy.py", line 193, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "/usr/lib/python3.8/runpy.py", line 86, in _run_code
        exec(code, run_globals)
      File "/home/viktor/scratch/bad_classmethod_as_documented.py", line 4, in <module>
        class Negator:
      File "/home/viktor/scratch/bad_classmethod_as_documented.py", line 12, in Negator
        def _(cls, arg: int):
      File "/usr/lib/python3.8/functools.py", line 906, in register
        return self.dispatcher.register(cls, func=method)
      File "/usr/lib/python3.8/functools.py", line 848, in register
        raise TypeError(
    TypeError: Invalid first argument to `register()`: <classmethod object at 0x7f37d1469070>. Use either `@register(some_class)` or plain `@register` on an annotated function.

Curiously, @staticmethod does work, but not as documented (don't decorate the actual implementations):

    from functools import singledispatchmethod
    
    
    class Negator:
        @singledispatchmethod
        @staticmethod
        def neg(arg):
            raise NotImplementedError("Cannot negate a")
    
        @neg.register
        def _(arg: int):
            return -arg
    
        @neg.register
        def _(arg: bool):
            return not arg
    
    
    if __name__ == "__main__":
        print(Negator.neg(0))
        print(Negator.neg(False))

Leads to

    $ python -m good_staticmethod
    0
    True

Removing @classmethod from the implementation methods doesn't work, though

    Traceback (most recent call last):
      File "/usr/lib/python3.8/runpy.py", line 193, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "/usr/lib/python3.8/runpy.py", line 86, in _run_code
        exec(code, run_globals)
      File "/home/viktor/scratch/bad_classmethod_alternative.py", line 20, in <module>
        print(Negator.neg(0))
      File "/usr/lib/python3.8/functools.py", line 911, in _method
        return method.__get__(obj, cls)(*args, **kwargs)
    TypeError: _() missing 1 required positional argument: 'arg'

[tirkarthi]

I guess the method checks for annotation on cls [0] which will be classmethod/staticmethod object in the report and won't have annotations. The annotations should be looked up in the function the classmethod/staticmethod decorator wraps around as in cls.__func__ . Something like below so that the right annotations are picked up. In addition to this the registry should store the type annotation as key to cls or cls.__func__ depending on normal method or classmethod/staticmethod.

diff --git Lib/functools.py Lib/functools.py
index 050bec8605..a66711208d 100644
--- Lib/functools.py
+++ Lib/functools.py
@@ -1073,24 +1073,33 @@ def singledispatch(func):
         if func is None:
             if isinstance(cls, type):
                 return lambda f: register(cls, f)
-            ann = getattr(cls, '__annotations__', {})
+            if isinstance(cls, (classmethod, staticmethod)):
+                ann = getattr(cls.__func__, '__annotations__', {})
+                func = cls.__func__
+            else:
+                ann = getattr(cls, '__annotations__', {})
+                func = cls

[0]

cpython/Lib/functools.py

Line 1076 in ab6423f

ann = getattr(cls, '__annotations__', {})

[vr2262]

I tried to apply this change but it didn't work, failing with this error

    $ ~/.pyenv/versions/3.8.1/bin/python -m bad_classmethod_as_documented
    Traceback (most recent call last):
      File "/home/viktor/.pyenv/versions/3.8.1/lib/python3.8/runpy.py", line 193, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "/home/viktor/.pyenv/versions/3.8.1/lib/python3.8/runpy.py", line 86, in _run_code
        exec(code, run_globals)
      File "/home/viktor/scratch/bad_classmethod_as_documented.py", line 4, in <module>
        class Negator:
      File "/home/viktor/scratch/bad_classmethod_as_documented.py", line 12, in Negator
        def _(cls, arg: int):
      File "/home/viktor/.pyenv/versions/3.8.1/lib/python3.8/functools.py", line 1006, in register
        return self.dispatcher.register(cls, func=method)
      File "/home/viktor/.pyenv/versions/3.8.1/lib/python3.8/functools.py", line 959, in register
        argname, cls = next(iter(get_type_hints(func).items()))
      File "/home/viktor/.pyenv/versions/3.8.1/lib/python3.8/typing.py", line 1252, in get_type_hints
        raise TypeError('{!r} is not a module, class, method, '
    TypeError: <classmethod object at 0x7f84e1e69c40> is not a module, class, method, or function.

After digging around a bit, this diff seems to work (not sure if there's a better way to do it) (also this one doesn't seem to care whether @staticmethod is applied to the implementation methods):

$ diff -u functools-orig.py functools.py
--- functools-orig.py	2020-02-21 16:14:56.141934001 -0500
+++ functools.py	2020-02-21 16:17:19.959905849 -0500
@@ -843,14 +843,18 @@
         if func is None:
             if isinstance(cls, type):
                 return lambda f: register(cls, f)
-            ann = getattr(cls, '__annotations__', {})
+            if isinstance(cls, (classmethod, staticmethod)):
+                ann = getattr(cls.__func__, '__annotations__', {})
+                func = cls.__func__
+            else:
+                ann = getattr(cls, '__annotations__', {})
+                func = cls
             if not ann:
                 raise TypeError(
                     f"Invalid first argument to `register()`: {cls!r}. "
                     f"Use either `@register(some_class)` or plain `@register` "
                     f"on an annotated function."
                 )
-            func = cls
 
             # only import typing if annotation parsing is necessary
             from typing import get_type_hints
@@ -908,6 +912,8 @@
     def __get__(self, obj, cls=None):
         def _method(*args, **kwargs):
             method = self.dispatcher.dispatch(args[0].__class__)
+            if isinstance(self.func, classmethod):
+                return method.__get__(obj, cls)(cls, *args, **kwargs)
             return method.__get__(obj, cls)(*args, **kwargs)
 
         _method.__isabstractmethod__ = self.__isabstractmethod__

[tirkarthi]

Sorry, I had the part only to detect annotations attached. My part is something similar to yours except it stores the appropriate function in the registry itself instead of passing the arguments at __get__ .

[vr2262]

Sorry to bump this after months of inactivity but is there something I need to do here?

[mgrandi]

same issue here, if we can't fix it then maybe we should edit the documentation to not suggest that @classmethod works?

[glyph]

I also just discovered this. I thought it must have regressed at some point but I see the docs say "new in 3.8" and I'm using 3.8.

Is there a "no CI for examples in the docs" issue that this could be linked to?