Fall back to .py files if stub for library module is missing

[JukkaL]

If a program imports a (standard) library module that has no stub, mypy should look at the module implementation, if available. I.e., mypy should follow PYTHONPATH by default.

We should ignore errors in Python std lib modules and just try to infer which variables, types, functions, methods, etc. are defined in the library so that we can do basic type checking (e.g., checking argument names and counts when calling module-level functions) and we can use classes defined in the module in annotations.

Prerequisites: Mypy should not crash or report a parse error for most Python 3.[234] standard library modules. There should a stub for most std lib C modules.

Open issues: What about errors in 3rd party library modules? Maybe we should silence them as well? We'd just infer as much as we can, but we probably shouldn't complain about them.

[gvanrossum]

I like the idea of not complaining about type errors in modules that are in the stdlib or 3rd party dependencies that were just installed, not written by the user running mypy. I'm not sure that $PYTHONPATH is how you determine that it's 3rd party code though. I also think you should probably at least log that you're doing this in case this causes degraded error reporting.

[JamesGuthrie]

I like this feature idea as it should make it a lot easier to get up and running with mypy. I would like to assist in the implementation, I'm quite new to mypy so I'm not sure if my assistance is more of a burden or a help. Could you give some hints as to how one would go about a concrete implementation?

I assume that the change would be something like: use the "current" mypy paths (pwd and $MYPYPATH) to determine if a stub/implementation exists, if not, fall back to $PYTHONPATH (or equivalent) and per default switch into an "ignore errors" mode for implementations which are in the $PYTHONPATH. I'm not sure yet how these details look internally in mypy, so having some tips as to where to look at the current implementation would be helpful.

It might be useful to add a --check-all (or similar) parameter to mypy to force checking through the entire codebase (including libs) and throw errors. It seems like a logical assumption that everything which is discovered by mypy in the $PYTHONPATH is a "library", but having the option to override that assumption surely isn't a bad thing.

[JukkaL]

Yes, your suggestion sounds reasonable. We'd report errors for files under MYPYPATH and those under the 'current directory', which is derived from the target file/module. If the target module can't be found via MYPYPATH, we'd fall back to PYTHONPATH (or sys.path), but we'd ignore (most) errors in these modules.

Here are some general pointers that may be helpful:

1. mypy.nodes.MypyFile should have a flag for whether we should ignore errors in the file or not.
2. mypy.build should fall back to PYTHONPATH and set the above MypyFile flag as needed after parsing a module.
3. mypy.errors.Errors should have a flag for ignoring errors, and we should set the flag during semantic analysis (mypy.semanal) and type checking (mypy.checker).

Also, we need to add a way of setting the ignore flag for a module in a test case. Detecting # type: ignore comment at module top-level would be a good way of testing the flag and setting it explicitly, but if you don't implement that yet, encoding the information in module name is a reasonable alternative (e.g., module name suffix _noerr could set the flag for the module in a test case).

Something like the --check-all flag is also a potentially good idea. Manually adding directories in PYTHONPATH to MYPYPATH is also an option (though less convenient).

Let me know if you'd like more specific pointers, and don't hesitate to ask questions if you get stuck :-)

[JamesGuthrie]

Related issues:
#175: Use PYTHONPATH to look up modules
#186: Do not type check modules that don't import typing
#562: Make it possible to ignore annotations in certain files/directories
#626: Allow using # type: ignore to skip type checking in a file

[JamesGuthrie]

It looks as though mypy already supports optionally using sys.path to search for modules through the --use-python-path flag.

I was playing around with that flag and came across something which is related to one component of #186. The example I was working with used the copy.copy() function, when mypy uses the system copy.py file it produces the error .../lib/python3.4/copy.py, line 61: No module named 'org.python.core' which appears to be related to jython.

The error is produced in a component of mypy.build which means that during the build process, if we try to import a module from within a module which is in the PYTHONPATH and it fails, we should ignore the failure.

[JukkaL]

Yeah, import errors should also be ignored in modules that we don't type check.

About #186: PEP 484 doesn't actually suggest looking at the existence of a typing import to determine if we should type check a module. Instead, mypy should probably not complain about top-level definitions, even if it can't infer a precise type, and should silently fall back to Any types. There could be a 'strict' type checking mode that complains about these things. For example, this should probably be okay, independent of whether the module imports typing:

x = []   # Fall back to List[Any]

This falling to Any would only happen for definitions at module top levels and within class bodies -- within statically typed functions we can complain if we can't infer a precise type. The difference is that a function annotation is a signal that we should check a function, but there is no such annotation mechanism for modules, so we should err on side of leniency.

In the strict type checking mode, we'd require an annotation for x in the above example.

[JukkaL]

Another idea is to automatically run stub generator on any modules in PYTHONPATH that don't have stubs. We'd cache the generated stub files somewhere such as a __mypy__ directory.

[ddfisher]

Less important now that we have --silent-imports.

[gvanrossum]

I'm not even sure I agree with this, if I understand the proposal correctly. How do we even know PYTHONPATH is relevant for the program? It's already used to find and run mypy; the program might use a different Python version (e.g. 2.7).

[JukkaL]

Yeah, using PYTHONPATH doesn't seem like the right way to approach this. But anyway, we could have another way of finding where the standard library lives for the target Python version/interpreter. However, it may be better to just use the stub generator to generate dynamically typed stubs for all standard library modules instead of trying to get mypy to process these modules.

[boompig]

I am not sure if related, but I am seeing this error with the very common yaml module.

[gvanrossum]

@boompig Can you be more specific what error you see? Preferably with a tiny example that we can repro.

[boompig]

Python file:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import print_function
import sys
import os.path
import logging
import json
from six.moves import input
from argparse import ArgumentParser
import subprocess
import time

import boto3
from botocore.exceptions import ClientError

import yaml

Contents of virtualenv from pip freeze:

appdirs==1.4.0
boto3==1.4.4
botocore==1.5.7
docutils==0.13.1
jmespath==0.9.1
mypy==0.470
packaging==16.8
pyflakes==1.5.0
pyparsing==2.1.10
python-dateutil==2.6.0
PyYAML==3.12
s3transfer==0.1.10
six==1.10.0
typed-ast==0.6.3

python version 3.5.1 on Mac OS X

Output of mypy test.py

$ mypy test.py
test.py:14: error: Cannot find module named 'boto3'
test.py:14: note: (Perhaps setting MYPYPATH or using the "--ignore-missing-imports" flag would help)
test.py:15: error: Cannot find module named 'botocore.exceptions'
test.py:17: error: No library stub file for module 'yaml'
test.py:17: note: (Stub files are from https://github.com/python/typeshed)

[gvanrossum]

OK, that's just because there are no stubs in typeshed for those modules yet (boto3, botocore.exceptions, and yaml -- the reason the latter has a different error message is that it's a known third party module). Feel free to submit a PR to add some or all of them to the typeshed repo.

In the meantime, do not attempt to point mypy to the site-packages directory. (See the note in the docs, at the end of this page.) Instead, use --ignore-missing-imports which will suppress such errors.

[justinpawela]

The note Guido mentioned above is lost to the ages, but here is its text:

You may be tempted to point MYPYPATH to the standard library or to the site-packages directory where your 3rd party packages are installed. This is almost always a bad idea – you will likely get tons of error messages about code you didn’t write and that mypy can’t analyze all that well yet, and in the worst case scenario mypy may crash due to some construct in a 3rd party package that it didn’t expect.

via the Internet Archive

[gvanrossum]

Actually the note just got moved; it's now on this page: http://mypy.readthedocs.io/en/latest/stubs.html

In any case I'm not sure that we should really do what's described at the top of this issue. For third party code we now have PEP 561 (and we're silencing errors there as of #5303).

[JelleZijlstra]

Can we just close this issue? I think PEP 561 solves this general area.

[gvanrossum]

Yeah, I don’t think we are going to implement the original idea.