Mypy issue validating package with directory and -p arg

[oleksiykamenyev]

Bug Report
I'm having an issue running mypy on a package with the -p argument where the package directory has a subdirectory inside it. I'm receiving the following error with mypy 0.9.10:

Traceback (most recent call last):
  File "/Users/oleksiyk/.pyenv/versions/3.8.9/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/oleksiyk/.pyenv/versions/3.8.9/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/__main__.py", line 23, in <module>
    console_entry()
  File "mypy/build.py", line 1955, in wrap_context
  File "mypy/build.py", line 2205, in finish_passes
  File "mypy/build.py", line 818, in report_file
  File "mypy/report.py", line 83, in file
  File "mypy/report.py", line 482, in on_file
  File "mypy/report.py", line 131, in iterate_python_lines
  File "/Users/oleksiyk/.pyenv/versions/3.8.9/lib/python3.8/tokenize.py", line 392, in open
    buffer = _builtin_open(filename, 'rb')
IsADirectoryError: [Errno 21] Is a directory: 'python_tools/resources'

I tested with the master branch, and the error is the same, but the traceback is different:

Traceback (most recent call last):
  File "/Users/oleksiyk/.pyenv/versions/3.8.9/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/oleksiyk/.pyenv/versions/3.8.9/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/__main__.py", line 23, in <module>
    console_entry()
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/__main__.py", line 11, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/main.py", line 87, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/main.py", line 165, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 179, in build
    result = _build(
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 254, in _build
    graph = dispatch(sources, manager, stdout)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 2707, in dispatch
    process_graph(graph, manager)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 3031, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 3132, in process_stale_scc
    graph[id].finish_passes()
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 2220, in finish_passes
    free_tree(self.tree)
  File "/Users/oleksiyk/.pyenv/versions/3.8.9/lib/python3.8/contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 1962, in wrap_context
    yield
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 2215, in finish_passes
    manager.report_file(self.tree, self.type_map(), self.options)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 821, in report_file
    self.reports.file(file, self.modules, type_map, options)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/report.py", line 85, in file
    reporter.on_file(tree, modules, type_map, options)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/report.py", line 483, in on_file
    for lineno, line_text in iterate_python_lines(path):
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/report.py", line 133, in iterate_python_lines
    with tokenize.open(path) as input_file:
  File "/Users/oleksiyk/.pyenv/versions/3.8.9/lib/python3.8/tokenize.py", line 392, in open
    buffer = _builtin_open(filename, 'rb')
IsADirectoryError: [Errno 21] Is a directory: 'python_tools/resources'

Without the -p argument, the command passes with no issue.

Expected Behavior

I'm not too familiar with mypy, but I would expect the code to ignore such directories I think. I'm also not sure why the -p argument would cause this issue, when the code otherwise works normally. When I dug through the traceback a bit, I found this function

mypy/mypy/report.py

Lines 121 to 128 in f98f782

	def should_skip_path(path: str) -> bool:
	if stats.is_special_module(path):
	return True
	if path.startswith('..'):
	return True
	if 'stubs' in path.split('/') or 'stubs' in path.split(os.sep):
	return True
	return False

that seems to handle skipping different paths for mypy validation. For the sake of testing, I changed the directory name above from resources to stubs, and that fixed the error. Perhaps all directories should be ignored by this skip function (or maybe they shouldn't be passed into it in the first place, not sure)?

My Environment

• Mypy version used: 0.9.10 and 0.920+dev.209a7193feb4bbfa38d09232b0a5a916e9d2e605
• Full command run: python3.8 -m mypy --txt-report ~/tmp/mypyr_report --install-types --non-interactive --namespace-packages --ignore-missing-imports -p yahoo.contrib.mx3_python_tools --show-traceback
• Mypy configuration options from mypy.ini (and other config files): none
• Python version used: 3.8.9 (although in a separate env, 3.9 showed the same issue)
• Operating system and version: MacOS Big Sur version 11.6

[jamesbraza]

I am getting a similar crash when using something like mypy -p pkg --html-report out with mypy==0.920 on a native namespace package.

Seems to be a bug in reporting related to -p

Traceback (most recent call last):
  File "/Users/james.braza/code/pkg/venv/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "mypy/build.py", line 1962, in wrap_context
  File "mypy/build.py", line 2215, in finish_passes
  File "mypy/build.py", line 821, in report_file
  File "mypy/report.py", line 85, in file
  File "mypy/report.py", line 483, in on_file
  File "mypy/report.py", line 133, in iterate_python_lines
  File "/Users/james.braza/.pyenv/versions/3.8.12/lib/python3.8/tokenize.py", line 392, in open
    buffer = _builtin_open(filename, 'rb')
IsADirectoryError: [Errno 21] Is a directory: 'pkg/path/to/dir'

[zevisert]

Also seeing this on 0.940+dev.14de8c6f1f24648299528881561ddcd52172701d.

I am blocked from using any reports, since my project is a namespace project that has found twice under different module names.

# mypy project --cobertura-xml-report mypy-reports --show-traceback
project/db/connect.py: error: Source file found twice under different module names: "connect" and "project.db.connect"
Found 1 error in 1 file (errors prevented further checking)

Traceback:

# mypy -p project --cobertura-xml-report mypy-reports --show-traceback 

/code/project/db: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.940+dev.14de8c6f1f24648299528881561ddcd52172701d
Traceback (most recent call last):
  File "/opt/tooling/pyenv/versions/hula-env/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/__main__.py", line 12, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/main.py", line 96, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/main.py", line 173, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 180, in build
    result = _build(
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 256, in _build
    graph = dispatch(sources, manager, stdout)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 2724, in dispatch
    process_graph(graph, manager)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 3068, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 3169, in process_stale_scc
    graph[id].finish_passes()
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 2209, in finish_passes
    with self.wrap_context():
  File "/opt/tooling/pyenv/versions/3.10.1/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 1970, in wrap_context
    yield
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 2225, in finish_passes
    manager.report_file(self.tree, self.type_map(), self.options)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 822, in report_file
    self.reports.file(file, self.modules, type_map, options)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/report.py", line 85, in file
    reporter.on_file(tree, modules, type_map, options)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/report.py", line 610, in on_file
    with tokenize.open(path) as input_file:
  File "/opt/tooling/pyenv/versions/3.10.1/lib/python3.10/tokenize.py", line 394, in open
    buffer = _builtin_open(filename, 'rb')
IsADirectoryError: [Errno 21] Is a directory: 'project/db'
/code/project/db: : note: use --pdb to drop into pdb
Generated Cobertura report: /code/project/mypy-reports/cobertura.xml