singledispatch on arguments that are themselves types/classes

[smheidrich]

Feature or enhancement

It would be great if functools.singledispatch supported dispatching on arguments that are themselves types / classes. E.g. this should be possible:

from functools import singledispatch

@singledispatch
def describe(x) -> str:
  raise TypeError(f"no description for {repr(x)}")

@describe.register
def _(x: type[int]) -> str:
  return "the integer type"

print(describe(int))

Currently, it just raises an exception saying Invalid annotation for 'x'. type[int] is not a class..

Pitch

The main argument for this, in my opinion, is that it's something that one would simply expect to work, given that we can use type[X] to refer to a subtype of X (rather than an instance of such a type) in other typing-related contexts.

You could object that one could make the same argument for types like list[int] which are also not currently supported by singledispatch, but in that case, anyone will understand after 5 seconds of thinking about it that it would require costly checks to determine whether an argument is of that type or not, so singledispatch can't easily support it. Meanwhile, the check for whether a given type is a subtype of a "simple" type like a class is at least in principle no more costly than the analogous check for an instance of that type, so one would expect it to just work.

Use cases

Here is one use case for this I've encountered in real life:

I often use singledispatch to define generic functions that transform (data)class instances to various representations of the contained data, e.g. to_json(obj), to_terminal_output(obj), and so on, the advantage over methods being that the classes themselves can be kept relatively "clean" and not concerned with the details of all these different formats. Naturally the question arises whether we could have similar functions for the inverse case, e.g. a generic function from_json(...) that can transform JSON back into any (data)class instance for which an implementation is provided. But how should we tell the generic function which class we want to deserialize to? If we want to stick with singledispatch, the natural way would be to simply have the class itself as the first argument (from_json(klass, json: str)) which is not currently possible as demonstrated in the example above.

Previous discussion

• https://discuss.python.org/t/singledispatch-on-arguments-that-are-themselves-types-classes/22264

Linked PRs

• GH-100623: Implement singledispatch on type/class arguments #100624

[smheidrich]

For anyone who needs/wants this earlier, I've created a standalone package which is just singledispatch with the patch from #100624 applied: https://gitlab.com/smheidrich/singledispatch-with-type-arg-support

[sobolevn]

I also maintain a separate similar library called dry-python/classes: https://github.com/dry-python/classes/ where we had the same discussion.

To sum up the long discussion we had:

• using the same method to register instances and types is confusing and can lead to errors
• @describe.register must also have a way to register the same function via argument, not just annotations
• implementation is quite brittle
• type checker support (mypy has a special plugin for this) will suffer
• there's no clean benefit, because one can use this hack:

  @describe.register
  def _(x: type) -> str:
     # Option 1:
     if hasattr(x, 'describe'):
        return x.describe()
     
     # Option 2:
     return GLOBAL_TYPE_DESCRIBE_REGISTRY.get(x, lambda: str(x))()

So, I am strongly -1 on this. Sorry!

[smheidrich]

@sobolevn Thanks for the detailed response! I think I agree with the outcome, but for completeness's sake, here are replies to every one of your points anyway - feel free to skip ahead to the last one which to me is the crux of the matter.

using the same method to register instances and types is confusing

You mean from the user's side or the implementation side? From the user's side, I think it's quite intuitive as long as we're using annotations, because type[x] is already familiar from other typing contexts (just as union annotations, which singledispatch supports, are intuitive for the same reason).

and can lead to errors

Such as? 🤔

@describe.register must also have a way to register the same function via argument, not just annotations

That is a good point and to be honest I can't think of a keyword that would transport the meaning of registering a function for type arguments rather than instance arguments... Something like register(type=...) or register(cls=...) wouldn't do because both can be interpreted as registering an implementation for instances of that type. Maybe register(type_itself=...)? 🙂

implementation is quite brittle

Hmm, not sure that has to be the case necessarily.

type checker support (mypy has a special plugin for this) will suffer

Perhaps, but as you've pointed out in https://sobolevn.me/2021/06/typeclasses-in-python#better-typing yourself, Mypy already can't handle a lot of singledispatch usages, e.g. it doesn't check if a given call will lead to a fallback implementation that raises NotImplementedError, can't handle TypeVars as in f(x: T) -> T (f being a singledispatch function) etc. So I guess this point is more about how much more difficult it would be to get all these things working with this feature implemented? Because at least my simple example above seems to be fine with current Mypy.

there's no clean benefit, because one can use this hack:

The 2nd option from your example got me thinking: All singledispatch is is a similar type -> implementation registry plus the ability to find the "most specific" implementation when only implementations for parent classes of the given argument's class exist. But considering e.g. the deserialization use case above, I don't think anyone would really use this latter feature: If you ask for from_json(MyClass, json_str), in the vast majority of cases you want exactly a MyClass instance. So perhaps singledispatch is just overkill for this use case, which was the main one I had in mind. 🤔

[smheidrich]

@sobolevn But by the way, in classes's introductory example for a typical (de)serialization helper class that classes seeks to make obsolete, you also have from_json, which is then not mentioned again - so you probably agree that there is a bit of a "gap" here, only filled by ad-hoc hacks like in your example?

[johnslavik]

I had a similar feature idea some time ago.

I must admit that whenever I had needed to make a singledispatch() out of types as arguments, I did the following in general...

my_dispatch = singledispatch(default_case_function)

def my_function(cls, *args, **kwargs):
    return my_dispatch.dispatch(cls)(cls, *args, **kwargs)

my_function.register = my_dispatch.register

No doubts that it's a bit of a workaround, and it works for me. But being honest, the presented idea of type[X] seems very intuitive and in line with the Liskov substitution principle. My linter doesn't like when I randomly set attributes on functions and want to use them in other places, guessing that same applies to type checkers. :P

[smheidrich]

Given the discussion both here and on discuss.python.org, looks like this won't make it in, at least not right now, so I'll go ahead and close both this issue and the PR.

@bswck Brilliant, I didn't even realize dispatch() was part of the public API. Seems like a decent workaround so long as you don't use annotations to register the individual implementations for different types (because if you did the annotation would be wrong) and document somehow that registering has a different meaning here (registering for arguments that themselves match the given type, not arguments whose type matches the type).

[johnslavik]

@smheidrich I've implemented your idea. https://github.com/bswck/class_singledispatch

[tillahoffmann]

Was looking for similar functionality recently and had success with the following hack. It instantiates a dummy object that returns the specified class when the __class__ attribute is accessed.

import functools
import numbers


def classdispatch(func):
    func = functools.singledispatch(func)
    
    def _inner(cls, *args, **kwargs):
        class Dummy:
            def __getattribute__(self, key):
                return cls
        return func(Dummy(), *args, **kwargs)

    _inner.register = func.register
    return _inner


@classdispatch
def do(cls):
    raise NotImplementedError


@do.register(numbers.Number)
def _(cls):
    print("a general number")


@do.register(int)
def _(cls):
    print("an integer")


do(int)  # an integer
do(float)  # a general number

[johnslavik]

Was looking for similar functionality recently and had success with the following hack. It instantiates a dummy object that returns the specified class when the __class__ attribute is accessed.

Orr you can use https://github.com/bswck/class_singledispatch :P