Wrong enum value type inferred for members defined in a subclass (regression)

[taljeth]

Bug Report

I have an Enum base class with a __new__ implementation with multiple parameters, so that for the declaration of members a tuple can be given, one of which is used as the value for the member. The members are only defined in a subclass (this is important for triggering the bug).

In mypy 0.790, the value of the members is still inferred as Any, whereas in mypy 0.800 it incorrectly infers it as the whole tuple given at the declaration (it should use the type of the one tuple item that is assigned to self._value_ in __new__()).

I have bisected the changed behaviour to commit 37777b3. Reverting only this commit on top of master is enough to get the old behaviour back.

To Reproduce

This can be reproduced with a slightly modified example from the Python documentation: https://docs.python.org/3/library/enum.html#when-to-use-new-vs-init

The modification is some added type hints and splitting the member definitions off into a child class (if they are defined in the same class, Any is still inferred).

from enum import Enum
from typing import Any
        
class Coordinate(bytes, Enum): 
    """ 
    Coordinate with binary codes that can be indexed by the int code.
    """ 
    def __init__(self, *args: Any) -> None:
        super().__init__()
        self.label: str
        self.unit: str
        self._value_: int
        
    def __new__(cls, value: int, label: str, unit: str) -> 'Coordinate':
        obj: Coordinate = bytes.__new__(cls, [value])
        obj._value_ = value
        obj.label = label
        obj.unit = unit
        return obj

class CoordinateSubclass(Coordinate):
    PX = (0, 'P.X', 'km')
    PY = (1, 'P.Y', 'km')
    VX = (2, 'V.X', 'km/s')
    VY = (3, 'V.Y', 'km/s')

c = CoordinateSubclass.PX
reveal_type(c)
reveal_type(c.value)

print(type(c))
print(type(c.value))
print("%d" % (c.value, ))

At runtime (after commenting out reveal_type), this gives the expected result:

<enum 'CoordinateSubclass'>
<class 'int'>
0

I'm running mypy with mypy --strict /tmp/enum_new.py, though strict isn't even needed for this one.

Expected Behavior

The old behaviour before commit 37777b3 is not ideal because it infers Any, but I consider this acceptable:

/tmp/enum_new.py:28: note: Revealed type is 'enum_new.CoordinateSubclass'
/tmp/enum_new.py:29: note: Revealed type is 'Any'

Even better behaviour would be inferring int for c.value.

Actual Behavior

After commit 37777b3, and in mypy 0.800, this is result instead:

/tmp/enum_new.py:28: note: Revealed type is 'enum_new.CoordinateSubclass'
/tmp/enum_new.py:29: note: Revealed type is 'Tuple[builtins.int, builtins.str, builtins.str]'
/tmp/enum_new.py:33: error: Incompatible types in string interpolation (expression has type "Tuple[int, str, str]", placeholder has type "Union[int, float, SupportsInt]")

The inferred Tuple type is just wrong.

Your Environment

• Mypy version used: 0.800 (and earlier versions for comparison and bisecting to commit 37777b3)
• Mypy command-line flags: none needed besides the filename
• Mypy configuration options from mypy.ini (and other config files): none
• Python version used: 3.9.1
• Operating system and version: Fedora 33

[taljeth]

I found some time to have a closer look today.

So it seems that the proper fix would be to teach mypy to infer attributes from __new__ (see #1021). Of course, this is somewhat out of scope for fixing the regression in #9443, and also for what I can realistically contribute.

Instead, for the time being I propose to stop trying to make a clever guess if the enum class in question implements __new__ because we just can't know what this implementation will do (if it exists, chances are that it's not the simple standard case).

The other surprising part was why the bug was only exposed on the subclass. The reason for this is that in the parent class, __init__ declares some instance attributes, and these are considered as possible enum values. This was enough to make the bug disappear because not all of the instance attributes have the same type in the example. Of course, this behaviour is wrong and only class attributes should be considered.

I'm going to prepare a PR to fix the problem along these lines.