python 3.7: coverage fails with TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType

[diefans]

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/olli/.pyenv/versions/3.7.0/envs/foobar/lib/python3.7/site-packages/_pytest/main.py", line 178, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/olli/.pyenv/versions/3.7.0/envs/foobar/lib/python3.7/site-packages/_pytest/main.py", line 215, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/home/olli/.pyenv/versions/3.7.0/envs/foobar/lib/python3.7/site-packages/pluggy/hooks.py", line 258, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/home/olli/.pyenv/versions/3.7.0/envs/foobar/lib/python3.7/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/olli/.pyenv/versions/3.7.0/envs/foobar/lib/python3.7/site-packages/pluggy/manager.py", line 61, in <lambda>
INTERNALERROR>     firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR>   File "/home/olli/.pyenv/versions/3.7.0/envs/foobar/lib/python3.7/site-packages/pluggy/callers.py", line 196, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/home/olli/.pyenv/versions/3.7.0/envs/foobar/lib/python3.7/site-packages/pytest_cov/plugin.py", line 228, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/home/olli/.pyenv/versions/3.7.0/envs/foobar/lib/python3.7/site-packages/pytest_cov/engine.py", line 167, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File "/home/olli/.pyenv/versions/3.7.0/envs/foobar/lib/python3.7/site-packages/coverage/control.py", line 781, in save
INTERNALERROR>     self.get_data()
INTERNALERROR>   File "/home/olli/.pyenv/versions/3.7.0/envs/foobar/lib/python3.7/site-packages/coverage/control.py", line 834, in get_data
INTERNALERROR>     self._post_save_work()
INTERNALERROR>   File "/home/olli/.pyenv/versions/3.7.0/envs/foobar/lib/python3.7/site-packages/coverage/control.py", line 849, in _post_save_work
INTERNALERROR>     self._warn_about_unmeasured_code(pkg)
INTERNALERROR>   File "/home/olli/.pyenv/versions/3.7.0/envs/foobar/lib/python3.7/site-packages/coverage/control.py", line 882, in _warn_about_unmeasured_code
INTERNALERROR>     has_file = hasattr(mod, '__file__') and os.path.exists(mod.__file__)
INTERNALERROR>   File "/home/olli/.pyenv/versions/3.7.0/lib/python3.7/genericpath.py", line 19, in exists
INTERNALERROR>     os.stat(path)
INTERNALERROR> TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType

In Python 3.7 namespace packages seem to have a __file__ == None attribute, which leads to coverage not recognizing it.

https://github.com/nedbat/coveragepy/blob/coverage-4.5.1x/coverage/control.py#L881

# ipython
Python 3.7.0 (default, Jul  1 2018, 11:34:46)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.5.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import com
In [2]: vars(com)
Out[2]:
{'__name__': 'com',
 '__doc__': None,
 '__package__': 'com',
 '__loader__': <_frozen_importlib_external._NamespaceLoader at 0x7f2944081588>,
 '__spec__': ModuleSpec(name='com', loader=<_frozen_importlib_external._NamespaceLoader object at 0x7f2944081588>, submodule_search_locations=_NamespacePath(['/home/code/ml/cookiecutter/foobar/src/com', '/home/olli/.pyenv/versions/3.7.0/envs/foobar/lib/python3.7/site-packages/com'])),
 '__file__': None,
 '__path__': _NamespacePath(['/home/code/ml/cookiecutter/foobar/src/com', '/home/olli/.pyenv/versions/3.7.0/envs/foobar/lib/python3.7/site-packages/com'])}
In [3]:

[nedbat]

@diefans Thanks for the report. Do you have a reproducible scenario I can try out?

[diefans]

Something like this:

(
export DIR=/tmp/test_cov_ns
rm -rf ${DIR}
mkdir -p ${DIR}/com/foo

cat > ${DIR}/com/foo/mod.py << EOF
def foobar():
    pass
EOF

cat > ${DIR}/test_cov.py << EOF
def test_com():
    import com.foo.mod
EOF
   
python3.7 -m venv ${DIR}/venv
${DIR}/venv/bin/pip install pytest coverage
cd ${DIR}; ${DIR}/venv/bin/coverage run --source com.foo ${DIR}/venv/bin/pytest test_cov.py
)

In my original scenario one dependency provides a part of the namespace and I call coverage via pytest --cov com to get the error, but this does not work in this simple case (who knows)...

[nedbat]

Thanks, this is a great reproducer :)

[diefans]

haha - finally I made someone happy ;)

[nedbat]

And it turns out this has already been fixed, in c9f4f66

[nedbat]

This fix is in 4.5.2, just released: https://pypi.org/project/coverage/4.5.2/