"Intermittent" import error when checking a module that import another module which imports a 3rd party package

[Quibi]

Hi,

Sorry for the title. I try to explain myself better.

I have found a strange behaviour in mypy related to an import error in a imported module. That is, an error in a module 'b' importing a module 'c' while checking module 'a', which imports 'b'. 'c', of course, is a 3rd party package.

In such a case, mypy reports nothing on a first round. But if it is run again, it reported a error: Cannot find module named 'c'. On a third run, nothing is reported again, while on a fourth run the error re-appears, and so on...

This happens even when the type check has been disabled in the import line.

A possible scenario to reproduce this "bug" is as follows:

1. Create a 'strange_report' package (with an empty __init__.py).

2. Create a subpackage called 'a' inside 'strange_report' (with an empty __init__.py too).

3. Write an a.py module (importer module) such as:

from a.b import B

b = B()  # just for visually check that a.py was run

4. Write a b.py module (imported module)

from ruamel.yaml import YAMLError  # type: ignore


class B:
    def __init__(self):
        print(f"New 'B' object {self} has been created")

5. Open terminal and change to directory 'strange_report'.

6. Module a.py can be run with python -m a.a

7. If you run mypy a/a.py for the first time, you do not get any reports (however, directory '.mypy_cache' is created).

8. But if you run mypy a/a.py once again, you get an error:

madera:strange_error jv$ mypy a/a.py
a/b.py:1: error: Cannot find module named 'ruamel'
a/a.py:1: error: Cannot find module named 'ruamel'
a/a.py:1: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports

9. You can repeat 7. and 8. again and again with similar results. Same results are gotten with or without # type: ignore.

However, if cache directory '.mypy_cache' is deleted before running mypy every time, it does not report anything.

Curiously enough, if you check mypy a/b.py it consistently reports nothing.

I am running on:

• macOS HIgh Sierra 10.13.6
• Python 3.7.4
• mypy 0.720
• ruamel.yaml 0.16.0 (so, yes, ruamel.yaml is installed; I use this 3rd party package because it is causing the bug in my system; no any particular purposes, I imagined that same bug can appeared with similar packages but cannot confirm that)
• Any mypy configuration beyond the default one.

Thank for your attention. I hope that will be helpful.

[ilevkivskyi]

Thanks for reporting! Does this also happen with --ignore-missing-imports?

[Quibi]

Oh, I am sorry. I missed your request.

mypy a/a.py --ignore-missing-imports does not raised the error.

Indeed, this is even more puzzling since according to documentation:

This flag makes mypy ignore all missing imports. It is equivalent to adding # type: ignore comments to all unresolved imports within your codebase.

Do not --ignore-missing-imports CLI argument and # type: ignore should cause the same error (or any error at all)?

Thank you.

[jsonmcfall]

We hit this error at least several times a day.
It seems to alway show up with:
AssertionError: Cannot find module for ruamel

[JukkaL]

Does using --no-incremental get rid of the issue?

[Quibi]

@jmcfallargo Of course, you must install ruamel.yaml first: https://pypi.org/project/ruamel.yaml/

It is just an example. I suspect that this issue can appear with other packages too, but I could not confirm that -- so I use this one in particular in my example.

[Quibi]

@JukkaL No issues with mypy a/a.py --no-incremental (in mypy 0.730). It is the same as deleting the cache by hand if I correctly understand the documentation. But this does not solve the issue, does not it?

Thank you.

[JukkaL]

It is the same as deleting the cache by hand if I correctly understand the documentation

Correct.

But this does not solve the issue, does not it?

Yeah, it doesn't solve the issue.

I'm increasing priority, at least until we better understand why (or how frequently) this happens, since incremental mode issues can be painful to diagnose and work around.

[Quibi]

Thank you, @JukkaL Sorry for not being able to provider further insights on this issue.

[asfaltboy]

I think what makes ruamel.yaml package "special" is that it uses the ruamel namespace (namely a directory in e.g site-packages with no __init__.py.

Could it be that we're not using the PEP 420 traversal (added by Barry Warsaw in PR #7219) since we're not targetting this package directly?

Back Ref: I had reported a similar issue with ruamel here: https://bitbucket.org/ruamel/yaml/issues/328/mypy-complains-that-ruamel-is-missing-type

[domdfcoding]

For future reference the issue @asfaltboy linked to is now at https://sourceforge.net/p/ruamel-yaml/tickets/328/