Dynamic base classes are not handled

[snakescott]

repro:

• set some $BASEDIR for files
• git clone https://github.com/python/typeshed.git $BASEDIR/typeshed
• cd $BASEDIR/typeshed; git checkout f36d7a3; rm -rf third_party/2/sqlalchemy/ (note: sqlalchemy stubs are incomplete, so ignore them)
• put the following code at $BASEDIR/failure.py

from sqlalchemy import Column, String
Base = declarative_base()

class Foo(Base):
  __tablename__ = 'foo'
  key = Column(String, primary_key=True)

• install mypy 0.4.6
• run mypy --py2 -v -v -v --fast-parser -s --custom-typeshed-dir $BASEDIR/typeshed $BASEDIR/failure.py

Output is something like

failure.py:33: error: Invalid type "failure.Base"
failure.py:33: error: Invalid base class

when it should be clean

[gvanrossum]

Simpler repro (without the typeshed/sqlalchemy stuff):

def func(): pass
Base = func()
class C(Base): pass  # errors on this line

Errors (mypy -s x.py):

x.py:3: error: Invalid type "x.Base"
x.py:3: error: Invalid base class

(Update: used def func... instead if import.)

[JukkaL]

The reason this doesn't work is that the type of Base is not yet inferred during semantic analysis, and that's when we process base classes. You can add an annotation as a workaround:

from typing import Any

def func(): pass
Base = func()  # type: Any
class C(Base): pass  # now works

[gvanrossum]

Should we close this then or update the docs somewhere? Maybe the error message could also be better.

[JukkaL]

I think that we should at least see if we can make the error message better.

[snakescott]

improved error message would be helpful. I did discover # type: Any as a workaround but it is good to be aware this is a known issue.

Is there a github issue I should be tracking for the semantic analysis improvement, or just watch the release notes?

[JukkaL]

Let's use this issue to track both the error message and the underlying semantic analysis issue for now.

[Deimos]

I seem to have hit a very similar bug that's a bit more convoluted (involving Optional and importing), but probably has a related base cause. It happened to me originally with sqlalchemy's declarative_base() as well, but this was my attempt to find a simple reproduction (Python 3.6):

file1.py:

from typing import Any

Base: Any

file2.py:

from typing import Optional
from .file1 import Base

class SubClass(Base):
    pass

def optional_return_func() -> Optional[SubClass]:
    return SubClass()

def should_fail() -> int:
    test_var = optional_return_func()
    reveal_type(test_var)

    if not test_var:
        return 1

    return 'not an int'

When I run mypy on this, the result is error: Revealed type is 'builtins.None'

The expected result should be Revealed type is 'Union[file2.SubClass, builtins.None]' along with an Incompatible return value type error, but since mypy is certain that test_var is None, it considers the final line that returns a string unreachable.

Hope that makes sense, let me know if any more information would be helpful.

[Deimos]

@JukkaL sorry for the ping, but I just wanted to check and see if you think the issue I described above is similar enough that leaving it as part of this one is good, or if I should open a new, separate issue for it.

[JukkaL]

@Deimos The builtins.None issue seems to be fixed on github master -- can you verify that it works for you? It's unrelated to the original issue, but no need to create an issue if it already works.

[Deimos]

Oh, yes, it does seem to be fixed using current github master. That's great, thanks (and sorry for bothering you about something that was already fixed).

[libre-man]

But if I add a Any annotation the problem is that it thinks the class can be anything, so doing stuff like

import typing as t
base: t.Any = make_base()
class A(base): pass

def make_list() -> t.Sequence[A]:
    return A()

does not yield an error which makes anything using this class (A in this case) not typesafe anymore.

[gvanrossum]

But what would you expect? Except for a few trivial edge cases, when the base class is computed dynamically, mypy is unable to understand the structure of your class. That's why it gives an error when you use a dynamic base class -- if you shut it up using Any, literally "Any"thing can happen. :-)

[I changed the issue topic because it has nothing to do with sqlalchemy.]