AssertionError when extending from an abstract model that includes a manager

[flapedriza]

Bug report

What's wrong

When running mypy with a model that extends from an abstract model that defines a manager, mypy crashes internally on the line where such model is defined.

The model (names are changed, but everything else is the same, the other abstract models only define fields):

class MyModel(SomeAbstractModel, OtherAbstractMode, SendableModel):
    field = models.BooleanField(default=False, db_index=True)
    
    def __str__(self) -> str:
        return f'Model {self.field}'

The abstract SendableModel model:

class SendableModel(UUIDModel):
    processing_started = models.DateTimeField(blank=False, null=True)
    sent = models.BooleanField(default=False, db_index=True)

    objects = models.Manager()
    to_send = ToSendManager()

    class Meta:
        abstract = True

I get the following traceback:

./users/models.py:72: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.rtfd.io/en/latest/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.770
Traceback (most recent call last):
  File "/usr/local/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/usr/local/lib/python3.6/site-packages/mypy/__main__.py", line 8, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/usr/local/lib/python3.6/site-packages/mypy/main.py", line 89, in main
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/usr/local/lib/python3.6/site-packages/mypy/build.py", line 181, in build
    sources, options, alt_lib_path, flush_errors, fscache, stdout, stderr, extra_plugins
  File "/usr/local/lib/python3.6/site-packages/mypy/build.py", line 249, in _build
    graph = dispatch(sources, manager, stdout)
  File "/usr/local/lib/python3.6/site-packages/mypy/build.py", line 2649, in dispatch
    process_graph(graph, manager)
  File "/usr/local/lib/python3.6/site-packages/mypy/build.py", line 2956, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/usr/local/lib/python3.6/site-packages/mypy/build.py", line 3049, in process_stale_scc
    mypy.semanal_main.semantic_analysis_for_scc(graph, scc, manager.errors)
  File "/usr/local/lib/python3.6/site-packages/mypy/semanal_main.py", line 78, in semantic_analysis_for_scc
    process_top_levels(graph, scc, patches)
  File "/usr/local/lib/python3.6/site-packages/mypy/semanal_main.py", line 203, in process_top_levels
    patches)
  File "/usr/local/lib/python3.6/site-packages/mypy/semanal_main.py", line 331, in semantic_analyze_target
    active_type=active_type)
  File "/usr/local/lib/python3.6/site-packages/mypy/semanal.py", line 350, in refresh_partial
    self.refresh_top_level(node)
  File "/usr/local/lib/python3.6/site-packages/mypy/semanal.py", line 361, in refresh_top_level
    self.accept(d)
  File "/usr/local/lib/python3.6/site-packages/mypy/semanal.py", line 4679, in accept
    node.accept(self)
  File "/usr/local/lib/python3.6/site-packages/mypy/nodes.py", line 939, in accept
    return visitor.visit_class_def(self)
  File "/usr/local/lib/python3.6/site-packages/mypy/semanal.py", line 1001, in visit_class_def
    self.analyze_class(defn)
  File "/usr/local/lib/python3.6/site-packages/mypy/semanal.py", line 1078, in analyze_class
    self.analyze_class_body_common(defn)
  File "/usr/local/lib/python3.6/site-packages/mypy/semanal.py", line 1087, in analyze_class_body_common
    self.apply_class_plugin_hooks(defn)
  File "/usr/local/lib/python3.6/site-packages/mypy/semanal.py", line 1147, in apply_class_plugin_hooks
    hook(ClassDefContext(defn, base_expr, self))
  File "/usr/local/lib/python3.6/site-packages/mypy_django_plugin/main.py", line 37, in transform_model_class
    process_model_class(ctx, django_context)
  File "/usr/local/lib/python3.6/site-packages/mypy_django_plugin/transformers/models.py", line 357, in process_model_class
    initializer_cls(ctx, django_context).run()
  File "/usr/local/lib/python3.6/site-packages/mypy_django_plugin/transformers/models.py", line 71, in run
    self.run_with_model_cls(model_cls)
  File "/usr/local/lib/python3.6/site-packages/mypy_django_plugin/transformers/models.py", line 231, in run_with_model_cls
    base_manager_info=manager_info)
  File "/usr/local/lib/python3.6/site-packages/mypy_django_plugin/transformers/models.py", line 186, in create_new_model_parametrized_manager
    method_node=sym.node)
  File "/usr/local/lib/python3.6/site-packages/mypy_django_plugin/lib/helpers.py", line 346, in copy_method_to_another_class
    assert bound_return_type is not None
AssertionError: 
./users/models.py:72: : note: use --pdb to drop into pdb

If I remove the objects and to_send definitions from SendableModel then mypy works correctly.

Edit: Just found out that if I remove the manager definition from the abstract model but put it in the final model, it still crashes

How is that should be

Mypy should run normally without crashing.

System information

• OS: Alpine 3.4 (docker)
• python version: 3.6
• django version: 2.2.6
• mypy version: 0.770 (also happens with 0.782)
• django-stubs version: 1.5.0

[kszmigiel]

Thanks for the report, even if it's a duplicate of #459
It's a known bug and I'm currently working on this.

[kszmigiel]

@flapedriza
Here's what I discovered.

models.py

from django.db import models

class SendableModel(models.Model):
    processing_started = models.DateTimeField(blank=False, null=True)
    sent = models.BooleanField(default=False, db_index=True)

    objects = models.Manager()

    class Meta:
        abstract = True


class MyModel(SendableModel):
    field = models.BooleanField(default=False, db_index=True)

    def __str__(self) -> str:
        return f'Model {self.field}'

views.py

from .models import MyModel

reveal_type(MyModel)
reveal_type(MyModel.objects)

When I run mypy --tb ./app/views.py I get:

polls/models.py:7: error: Need type annotation for 'objects'
polls/views.py:3: note: Revealed type is 'def (*args: Any, **kwargs: Any) -> polls.models.MyModel'
polls/views.py:4: note: Revealed type is 'django.db.models.manager.Manager[polls.models.MyModel]'

I guess we can consider this as correct behavior, can we @sobolevn?

First error can be easily fixed by adding type annotation to objects field:

objects : models.Manager = models.Manager()

@flapedriza could you try to use Mypy on your code with my version of Django-Stubs? It can be found here: https://github.com/kszmigiel/django-stubs/tree/feature/plugin_cleanup
Checkout to branch feature/plugin_cleanup and do pip install -e .. It'll install "almost done django-stubs@1.6.0" on your machine.

[flapedriza]

Hi @kszmigiel, thanks for your answer.

I did what you suggested (although I changed the setup.py file to use django 2.2.6 before doing pip install -e . because my code is not compatible with django 3.1).

Now the problem mentioned before does not cause a crash, but it crashes for another reason.

Here's the traceback:

/code # mypy . --tb
common/base_models.py:34: error: Missing type parameters for generic type "Manager"
common/base_models.py:61: error: Need type annotation for 'objects'
users/models.py:40: error: Name 'QuerySet' is not defined
./users/models.py:135: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.rtfd.io/en/latest/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.782
Traceback (most recent call last):
  File "/usr/local/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/usr/local/lib/python3.6/site-packages/mypy/__main__.py", line 8, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/usr/local/lib/python3.6/site-packages/mypy/main.py", line 89, in main
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/usr/local/lib/python3.6/site-packages/mypy/build.py", line 181, in build
    sources, options, alt_lib_path, flush_errors, fscache, stdout, stderr, extra_plugins
  File "/usr/local/lib/python3.6/site-packages/mypy/build.py", line 252, in _build
    graph = dispatch(sources, manager, stdout)
  File "/usr/local/lib/python3.6/site-packages/mypy/build.py", line 2626, in dispatch
    process_graph(graph, manager)
  File "/usr/local/lib/python3.6/site-packages/mypy/build.py", line 2949, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/usr/local/lib/python3.6/site-packages/mypy/build.py", line 3047, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/usr/local/lib/python3.6/site-packages/mypy/build.py", line 2107, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/usr/local/lib/python3.6/site-packages/mypy/checker.py", line 294, in check_first_pass
    self.accept(d)
  File "/usr/local/lib/python3.6/site-packages/mypy/checker.py", line 401, in accept
    stmt.accept(self)
  File "/usr/local/lib/python3.6/site-packages/mypy/nodes.py", line 939, in accept
    return visitor.visit_class_def(self)
  File "/usr/local/lib/python3.6/site-packages/mypy/checker.py", line 1724, in visit_class_def
    self.accept(defn.defs)
  File "/usr/local/lib/python3.6/site-packages/mypy/checker.py", line 401, in accept
    stmt.accept(self)
  File "/usr/local/lib/python3.6/site-packages/mypy/nodes.py", line 1004, in accept
    return visitor.visit_block(self)
  File "/usr/local/lib/python3.6/site-packages/mypy/checker.py", line 1973, in visit_block
    self.accept(s)
  File "/usr/local/lib/python3.6/site-packages/mypy/checker.py", line 401, in accept
    stmt.accept(self)
  File "/usr/local/lib/python3.6/site-packages/mypy/nodes.py", line 766, in accept
    return visitor.visit_decorator(self)
  File "/usr/local/lib/python3.6/site-packages/mypy/checker.py", line 3526, in visit_decorator
    self.check_func_item(e.func, name=e.func.name)
  File "/usr/local/lib/python3.6/site-packages/mypy/checker.py", line 792, in check_func_item
    self.check_func_def(defn, typ, name)
  File "/usr/local/lib/python3.6/site-packages/mypy/checker.py", line 975, in check_func_def
    self.accept(item.body)
  File "/usr/local/lib/python3.6/site-packages/mypy/checker.py", line 401, in accept
    stmt.accept(self)
  File "/usr/local/lib/python3.6/site-packages/mypy/nodes.py", line 1004, in accept
    return visitor.visit_block(self)
  File "/usr/local/lib/python3.6/site-packages/mypy/checker.py", line 1973, in visit_block
    self.accept(s)
  File "/usr/local/lib/python3.6/site-packages/mypy/checker.py", line 401, in accept
    stmt.accept(self)
  File "/usr/local/lib/python3.6/site-packages/mypy/nodes.py", line 1062, in accept
    return visitor.visit_assignment_stmt(self)
  File "/usr/local/lib/python3.6/site-packages/mypy/checker.py", line 2011, in visit_assignment_stmt
    self.check_assignment(s.lvalues[-1], s.rvalue, s.type is None, s.new_syntax)
  File "/usr/local/lib/python3.6/site-packages/mypy/checker.py", line 2147, in check_assignment
    rvalue_type = self.expr_checker.accept(rvalue)
  File "/usr/local/lib/python3.6/site-packages/mypy/checkexpr.py", line 3761, in accept
    typ = node.accept(self)
  File "/usr/local/lib/python3.6/site-packages/mypy/nodes.py", line 1834, in accept
    return visitor.visit_super_expr(self)
  File "/usr/local/lib/python3.6/site-packages/mypy/checkexpr.py", line 3477, in visit_super_expr
    in_literal_context=self.is_literal_context())
  File "/usr/local/lib/python3.6/site-packages/mypy/checkmember.py", line 126, in analyze_member_access
    result = _analyze_member_access(name, typ, mx, override_info)
  File "/usr/local/lib/python3.6/site-packages/mypy/checkmember.py", line 143, in _analyze_member_access
    return analyze_instance_member_access(name, typ, mx, override_info)
  File "/usr/local/lib/python3.6/site-packages/mypy/checkmember.py", line 225, in analyze_instance_member_access
    return analyze_member_var_access(name, typ, info, mx)
  File "/usr/local/lib/python3.6/site-packages/mypy/checkmember.py", line 376, in analyze_member_var_access
    return analyze_var(name, v, itype, info, mx, implicit=implicit)
  File "/usr/local/lib/python3.6/site-packages/mypy/checkmember.py", line 603, in analyze_var
    result, mx.context, mx.chk))
  File "/build/django-stubs/mypy_django_plugin/lib/helpers.py", line 321, in __call__
    assert isinstance(ctx.context, MemberExpr)
AssertionError: 
./users/models.py:135: : note: use --pdb to drop into pdb

This seems to be caused when calling super() in a property inside a model, as the line of code that is crashing is the following:

And if I change it to ret = {} or use a function instead of a property it does not crash.

On the other hand, there's 2 warnings that confuse me a bit:

• common/base_models.py:34: error: Missing type parameters for generic type "Manager": This line of code is a class definition (class ToSendManager(models.Manager):), and the defined class inherits from Manager (maybe it's getting confused between the inheritance and a type annotation there?)
• users/models.py:40: error: Name 'QuerySet' is not defined: In that line of code there's the following:

  class Meta:
      abstract = True

  And the word "Queryset" does not appear in the full file.

Should I open a new Issue with the problems I stated before?

Thanks,

[kszmigiel]

@flapedriza There's no need for another issue. Could you please provide some minimum reproduction repository for these errors? It'd help a lot.

[flapedriza]

Hi @kszmigiel sorry for the delay.

After testing again, it seems that the crash is not happening anymore.

Both other strange errors are still showing though.

Here's my mypy config:

[mypy]
plugins =
    mypy_django_plugin.main,
    mypy_drf_plugin.main
disallow_untyped_defs = True
strict = True

[mypy-coverage.*]
ignore_missing_imports = True

[mypy-gnupg.*]
ignore_missing_imports = True

[mypy-django_countries.*]
ignore_missing_imports = True

[mypy-phonenumber_field.*]
ignore_missing_imports = True

[mypy-prometheus_client.*]
ignore_missing_imports = True

[mypy-phonenumbers.*]
ignore_missing_imports = True

[mypy-django_q.*]
ignore_missing_imports = True

[mypy-paramiko.*]
ignore_missing_imports = True

[mypy-*.migrations.*]
ignore_errors = True

[mypy-*.tests.*]
ignore_errors = True

[mypy-*.admin.*]
ignore_errors = True

[mypy.plugins.django-stubs]
django_settings_module = "program.settings.test"

And this file has the same behavior as the original code:

import logging
from typing import Any, Dict

from django.db import models
from django.db.models import QuerySet

logger = logging.getLogger(__name__)


class TimestampedModel(models.Model):
    created = models.DateTimeField(auto_now_add=True, editable=False)
    modified = models.DateTimeField(auto_now=True)

    class Meta:
        abstract = True


class ToSendManager(
    models.Manager
):  # <Missing type parameters for generic type "Manager"> (using strict = True) should not happen
    # as this is not an annotation
    def get_queryset(self) -> 'QuerySet[models.Model]':
        qs = super().get_queryset()
        return qs.filter(processing_started__isnull=True, sent=False)


class SendableModel(models.Model):
    processing_started = models.DateTimeField(blank=False, null=True)
    sent = models.BooleanField(default=False, db_index=True)

    objects: 'models.Manager[SendableModel]' = models.Manager()
    to_send = ToSendManager()

    class Meta:
        abstract = True


class LoyaltyAbstractUser(TimestampedModel):
    email = models.EmailField(unique=True, blank=False, null=False)

    # Workaround to use custom Token authentication
    is_active = True
    is_authenticated = True

    class Meta:  # This is were <error: Name 'QuerySet' is not defined> appears in the original code
                 # but I'm not able to reproduce here
        abstract = True


class AbstractLogin(models.Model):
    @property
    def payload(self) -> Dict[str, Any]:
        return {}

    class Meta:
        abstract = True


class AircraftLogin(AbstractLogin):
    user_uuid = models.CharField(null=False, blank=False, editable=False, max_length=36)

    @property
    def payload(self) -> Dict[str, Any]:
        ret = super(AircraftLogin, self).payload  # This is where it crashed
                                                  # (now it seems that it is not crashing)
        return ret


class FailedLogin(AbstractLogin):
    email = models.EmailField(blank=False, null=False)

    @property
    def payload(self) -> Dict[str, Any]:
        ret = super(FailedLogin, self).payload
        return ret

I was not able to reproduce the Name 'QuerySet' is not defined error. But I'll try to create a repo where it happens too.

[kszmigiel]

@flapedriza thank you! I'll get back to this soon.

[kszmigiel]

PR's are welcome too!

[flaeppe]

I think this should be closed via #1169 since it includes a traceback of copy_method_to_another_class. Which was removed in there.