mypy --follow-imports error or skip crashes with numpy 2.0.0

[tomsilver]

This is a mypy crash revealed by the release of numpy 2.0.0. See also numpy/numpy#26720

Crash Report

The crash is shown below. Notes:

• The crash does not happen with numpy==1.26.4
• The same crash happens with mypy test.py --follow-imports skip
• It does not happen with mypy test.py --follow-imports normal

Traceback

Traceback (most recent call last):
  File "/Users/tom/Desktop/venv/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/Users/tom/Desktop/venv/lib/python3.10/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "mypy/main.py", line 100, in main
  File "mypy/main.py", line 182, in run_build
  File "mypy/build.py", line 192, in build
  File "mypy/build.py", line 266, in _build
  File "mypy/build.py", line 2942, in dispatch
  File "mypy/build.py", line 3340, in process_graph
  File "mypy/build.py", line 3467, in process_stale_scc
  File "mypy/build.py", line 2503, in write_cache
  File "mypy/build.py", line 1564, in write_cache
  File "mypy/nodes.py", line 387, in serialize
  File "mypy/nodes.py", line 3936, in serialize
  File "mypy/nodes.py", line 3873, in serialize
  File "mypy/nodes.py", line 3304, in serialize
  File "mypy/types.py", line 667, in serialize
  File "mypy/types.py", line 2430, in serialize
  File "mypy/types.py", line 1468, in serialize
  File "mypy/types.py", line 667, in serialize
  File "mypy/types.py", line 3067, in serialize
AssertionError: Internal error: unresolved placeholder type None

To Reproduce

1. pip install numpy mypy
2. echo "import numpy" > test.py
3. mypy test.py --follow-imports error

Your Environment

• Mypy version used: mypy 1.10.0 (compiled: yes)
• Mypy command-line flags: --follow-imports error
• Mypy configuration options from mypy.ini (and other config files): N/A
• Python version used: 3.10.6
• Operating system and version: MacOS Monterey (12.5.1)

[fynnsu]

I also got the same error running stubgen on a package, when an (non-direct) ancestor dependency used numpy>=2.0.

[hauntsaninja]

Thanks for the issue.

I minimised the repro, in case someone has time to look into it:

rm -rf repro
mkdir repro
mkdir repro/np
echo 'from .arraysetops import UniqueAllResult' > repro/np/__init__.pyi
echo '
from typing import Generic, NamedTuple, TypeVar
from np import does_not_exist
from np.missing import also_missing

_SCT = TypeVar("_SCT", bound=does_not_exist)

class UniqueAllResult(NamedTuple, Generic[_SCT]):
    values: also_missing[_SCT]
' > repro/np/arraysetops.pyi
touch repro/np/py.typed

PYTHONPATH=repro mypy -c 'import np'

[hauntsaninja]

(also standard disclaimer: a global --follow-imports flag is fairly niche and in practice I've seen more misuse of it than legitimate use)

[rgommers]

Thank you for looking into this @hauntsaninja. Would you mind clarifying what does_not_exist and also_missing are in the numpy stub file? It looks from your reproducer that there's something to fix there for us in numpy, and it has to do with numpy.generic, numpy.number, typing.Generic and/or numpy._typing.NDArray`. But they all seem to exist as far as I can tell.

[hauntsaninja]

@rgommers I don't think there's anything for numpy to do here, this just needs to be fixed in mypy. --follow-imports error is an unusual setting that makes mypy not resolve imports.

That said, there is one unrelated case of a missing import in numpy's type stubs that would be good to fix:

numpy/ma/extras.pyi:2: error: Cannot find implementation or library stub for module named "numpy.lib.index_tricks"  [import-not-found]

[rgommers]

Thanks @hauntsaninja. I opened a PR to fix that issue, it should land in numpy 2.0.1.

[owillebo]

Running stubgen over from numpy import array also fails with this error.

Running attached test.txt Windows batch script results in;

Python 3.11.9
mypy==1.10.0
mypy-extensions==1.0.0
numpy==2.0.0
typing_extensions==4.12.2
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Temp\mypy\.venv\Scripts\stubgen.exe\__main__.py", line 7, in <module>
  File "mypy\stubgen.py", line 1882, in main
  File "mypy\stubgen.py", line 1678, in generate_stubs
  File "mypy\stubgen.py", line 1610, in generate_asts_for_modules
  File "mypy\build.py", line 192, in build
  File "mypy\build.py", line 266, in _build
  File "mypy\build.py", line 2942, in dispatch
  File "mypy\build.py", line 3340, in process_graph
  File "mypy\build.py", line 3467, in process_stale_scc
  File "mypy\build.py", line 2503, in write_cache
  File "mypy\build.py", line 1564, in write_cache
  File "mypy\nodes.py", line 387, in serialize
  File "mypy\nodes.py", line 3936, in serialize
  File "mypy\nodes.py", line 3873, in serialize
  File "mypy\nodes.py", line 3304, in serialize
  File "mypy\types.py", line 667, in serialize
  File "mypy\types.py", line 2430, in serialize
  File "mypy\types.py", line 1468, in serialize
  File "mypy\types.py", line 667, in serialize
  File "mypy\types.py", line 3067, in serialize
AssertionError: Internal error: unresolved placeholder type None

[Inkaros]

If this helps at all, stubgen from mypy < 1.5.0 does not produce this error when generating stubs for modules that use numpy 2.0.0.

[LordAro]

Following the hint from @Inkaros, I've bisected the issue to b995e16

(with test.py from numpy import array & running pip install -e . && stubgen test.py)

I don't understand mypy well enough to take it much further though