build: log build sources with -v

[hauntsaninja]

With the changes I've been making to mypy's import handling, I think
this would be a useful thing to add preemptively.

Note that I would have found this very useful at points, and I think
others would too, eg #7672 and #8584

The existing logging ignores source_modules and source_text and doesn't
help with determining what mypy things the module name for a given file
is. This is useful for namespace package issues as in the complaint in #8584.

[gvanrossum]

Hmm, I see that when I turn this on to mypy itself, it claims that some sources from typeshed/stdlib/2 and typeshed/thir_party/2 are found, e.g.

LOG:  Found source:           BuildSource(path='mypy/typeshed/stdlib/2/distutils/__init__.pyi', module='distutils', has_text=False, base_dir='mypy/typeshed/stdlib/2')
LOG:  Found source:           BuildSource(path='mypy/typeshed/stdlib/2/distutils/command/__init__.pyi', module='distutils.command', has_text=False, base_dir='mypy/typeshed/stdlib/2')
LOG:  Found source:           BuildSource(path='mypy/typeshed/stdlib/2/distutils/command/bdist.pyi', module='distutils.command.bdist', has_text=False, base_dir='mypy/typeshed/stdlib/2')
.
.
.
LOG:  Found source:           BuildSource(path='mypy/typeshed/third_party/2/ipaddress.pyi', module='ipaddress', has_text=False, base_dir='mypy/typeshed/third_party/2')
LOG:  Found source:           BuildSource(path='mypy/typeshed/third_party/2/pathlib2.pyi', module='pathlib2', has_text=False, base_dir='mypy/typeshed/third_party/2')
LOG:  Found source:           BuildSource(path='mypy/typeshed/third_party/2/pymssql.pyi', module='pymssql', has_text=False, base_dir='mypy/typeshed/third_party/2')

(A total of 265 files, out of 1436 reported.)

Any idea what's up with that?

[gvanrossum]

Hm, I'm guessing this is the new, improved file search? This was from

python -m mypy -v mypy

It does the right thing when I run

python -m mypy -v -p mypy

IIRC the former used to work. Well, I guess I never liked typeshed being embedded inside mypy (nor the test subdir for that matter), but here we are.

[gvanrossum]

IOW I realize I have to declare in favor of this PR, since this actually showed the error of my ways... :-)

[hauntsaninja]

:-)
I also noticed the error message isn't accurate, for which I made #9674!

[gvanrossum]

But I still want a -v flag that just prints the list of sources, and not all the other stuff that’s logged by -v. That could become -vv.

Also, wouldn’t it be nice if each source file was logged just before it got checked, instead of all ahead of the checking? Then it would intersperse this activity log with the errors, as found.

[hauntsaninja]

Yeah that would be nice, and there's still the "show all files reached by following imports" part of your original request.

It also occurred to me that python -m mypy --config-file mypy_self_check.ini -p mypy --namespace-packages -v should also end up checking all of typeshed, but it doesn't. So #9616 isn't quite the fix I thought it was...

[gvanrossum]

Presuming you meant ‘-p mypy’, maybe it just stops descending when it finds a package named ‘mypy’?

[hauntsaninja]

Yeah, I meant -p mypy (edited above). The issue is that modulefinder recurses differently in the presence of __init__.py; we hit this check even if we allow namespace packages, which is not what we want

[samskiter]

Also, wouldn’t it be nice if each source file was logged just before it got checked, instead of all ahead of the checking? Then it would intersperse this activity log with the errors, as found.

I know you posted this a while ago but definitely a big YES to this. - Currently trying to help fix this VSCode mypy extension and that feature would come in might handy for this bug microsoft/vscode-mypy#123

Any suggestions for a quick fix? If the output can include files checked, even if they have no error, it makes clearing errors on files a lot easier...