report.py Tries to parse non-existent files

[adam-grant-hendry]

Describe the bug

After uninstalling PyQt5 and installing PySide6 in my virtual environment (using poetry add/remove), coverage reports warnings that it cannot parse some non-existent files. It is looking for site-packages pyscript, shibokensupport, and signature_bootstrap.py as if they are in my local folder, when they are actually in my .venv/Lib/site-packages folder:

myproj\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse 'Myproj\pyscript': No source for code: 'Myproj\pyscript'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
myproj\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse 'Myproj\shibokensupport\__init__.py': No source for code: 'Myproj\shibokensupport\__init__.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
myproj\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse 'Myproj\shibokensupport\feature.py': No source for code: 'Myproj\shibokensupport\feature.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
myproj\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse 'Myproj\shibokensupport\signature\__init__.py': No source for code: 'Myproj\shibokensupport\signature\__init__.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
myproj\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse 'Myproj\shibokensupport\signature\errorhandler.py': No source for code: 'Myproj\shibokensupport\signature\errorhandler.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
myproj\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse 'Myproj\shibokensupport\signature\importhandler.py': No source for code: 'Myproj\shibokensupport\signature\importhandler.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
myproj\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse 'Myproj\shibokensupport\signature\layout.py': No source for code: 'Myproj\shibokensupport\signature\layout.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
myproj\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse 'Myproj\shibokensupport\signature\lib\__init__.py': No source for code: 'Myproj\shibokensupport\signature\lib\__init__.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
myproj\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse 'Myproj\shibokensupport\signature\lib\enum_sig.py': No source for code: 'Myproj\shibokensupport\signature\lib\enum_sig.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
myproj\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse 'Myproj\shibokensupport\signature\lib\pyi_generator.py': No source for code: 'Myproj\shibokensupport\signature\lib\pyi_generator.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
myproj\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse 'Myproj\shibokensupport\signature\lib\tool.py': No source for code: 'Myproj\shibokensupport\signature\lib\tool.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
myproj\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse 'Myproj\shibokensupport\signature\loader.py': No source for code: 'Myproj\shibokensupport\signature\loader.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
myproj\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse 'Myproj\shibokensupport\signature\mapping.py': No source for code: 'Myproj\shibokensupport\signature\mapping.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
myproj\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse 'Myproj\shibokensupport\signature\parser.py': No source for code: 'Myproj\shibokensupport\signature\parser.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
myproj\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse 'Myproj\signature_bootstrap.py': No source for code: 'Myproj\signature_bootstrap.py'. (couldnt-parse)       
  coverage._warn(msg, slug="couldnt-parse")

To Reproduce
How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:

1. What version of Python are you using? 3.8.10
2. What version of coverage.py shows the problem? The output of coverage debug sys is helpful. 6.4
3. What versions of what packages do you have installed? The output of pip freeze is helpful.

[tool.poetry.dependencies]
appdirs = "^1.4.4"
h5py = "^3.6.0"
hypothesis = "^6.46.2"
ipython = "^8.3.0"
matplotlib = "^3.5.2"
merry = "^0.3.0"
meshio = "^5.3.4"
ninja = "^1.10.2"
numpy = "^1.22.3"
opencv-python = "^4.5.5.64"
pyembree = "^0.2.11"
pykdtree = "^1.3.4"
pymeshfix = "^0.15.0"
pyqtgraph = "^0.12.4"
PySide6 = "^6.3.0"
pytest-xdist = "^2.5.0"
python = "3.8"
pyvista = "^0.34.1"
pyvistaqt = "^0.9.0"
QtPy = "^2.1.0"
Rtree = "^1.0.0"
trimesh = "^3.11.2"
vtk = "^9.1.0"

[tool.poetry.dev-dependencies]
bandit = {extras = ["toml"], version = "^1.7.4"}
beautifulsoup4 = "4.11.1"
black = "^22.3.0"
bump2version = "^1.0.1"
check-jsonschema = "^0.14.3"
codecov = "^2.1.12"
codespell = "^2.1.0"
commitizen = "^2.27.0"
Cython = "^0.29.28"
docformatter = "^1.4"
flake8 = "^4.0.1"
flake8-docstrings = "^1.6.0"
flake8-quotes = "^3.3.1"
html5lib = "^1.1"
isort = "^5.10.1"
lxml = "^4.8.0"
memory-profiler = "^0.60.0"
MonkeyType = "^22.2.0"
mypy = "^0.950"
myst-parser = "^0.17.2"
pep8-naming = "^0.12.1"
pre-commit = "^2.18.1"
pydocstyle = "^6.1.1"
pyinstaller = "^5.1"
pylint = "^2.13.8"
pytest = "^7.1.2"
pytest-doctestplus = "^0.12.0"
pytest-env = "^0.6.2"
pytest-memprof = "^0.2.0"
pytest-qt = "^4.0.2"
pytest-randomly = "^3.11.0"
requests = "^2.27.1"
rstcheck = "^5.0.0"
Sphinx = "^4.5.0"
sphinx-rtd-theme = "^1.0.0"
sphinxcontrib-email = "^0.3.5"
sphinxcontrib-napoleon = "^0.7"
toml = "^0.10.2"
tox = "^3.25.0"
tqdm = "^4.64.0"
vulture = "^2.3"
wheel = "^0.37.1"
pytest-cov = "^3.0.0"
coverage = {extras = ["toml"], version = "^6.4"}

4. What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix.
   Not a specific code example as this is an internal warning.

5. What commands did you run?

pytest --cov --cov-report term-missing --cov-report html --cov-report xml

Expected behavior
A clear and concise description of what you expected to happen.
These warnings should not occur because these are non-existent file paths.

Additional context
Add any other context about the problem here.

I deleted every single __pycache__ and *.pyc, as well as the .coverage file in the project, but the problem persists.

[nedbat]

I'm not sure how to help without a way to reproduce this. Can you try setting COVERAGE_DEBUG=trace in your environment? It will show tracing decisions that might provide clues.

[adam-grant-hendry]

@nedbat Yes, I'll do this. One sec.

[adam-grant-hendry]

I've attached the debug log. As you can see, it is trying to trace files that don't exist (they are not in myproj):

Tracing 'pyscript' as '%USERPROFILE%\\Code\\myproj\\pyscript'
Tracing 'signature_bootstrap.py' as '%USERPROFILE%\\Code\\myproj\\signature_bootstrap.py'
Tracing 'shibokensupport/__init__.py' as '%USERPROFILE%\\Code\\myproj\\shibokensupport\\__init__.py'
Tracing 'shibokensupport/signature/__init__.py' as '%USERPROFILE%\\Code\\myproj\\shibokensupport\\signature\\__init__.py'
Tracing 'shibokensupport/signature/loader.py' as '%USERPROFILE%\\Code\\myproj\\shibokensupport\\signature\\loader.py'
Tracing 'shibokensupport/feature.py' as '%USERPROFILE%\\Code\\myproj\\shibokensupport\\feature.py'
Tracing 'shibokensupport/signature/mapping.py' as '%USERPROFILE%\\Code\\myproj\\shibokensupport\\signature\\mapping.py'
Tracing 'shibokensupport/signature/errorhandler.py' as '%USERPROFILE%\\Code\\myproj\\shibokensupport\\signature\\errorhandler.py'
Tracing 'shibokensupport/signature/layout.py' as '%USERPROFILE%\\Code\\myproj\\shibokensupport\\signature\\layout.py'
Tracing 'shibokensupport/signature/lib/__init__.py' as '%USERPROFILE%\\Code\\myproj\\shibokensupport\\signature\\lib\\__init__.py'
Tracing 'shibokensupport/signature/parser.py' as '%USERPROFILE%\\Code\\myproj\\shibokensupport\\signature\\parser.py'
Tracing 'shibokensupport/signature/lib/tool.py' as '%USERPROFILE%\\Code\\myproj\\shibokensupport\\signature\\lib\\tool.py'
Tracing 'shibokensupport/signature/importhandler.py' as '%USERPROFILE%\\Code\\myproj\\shibokensupport\\signature\\importhandler.py'
Tracing 'shibokensupport/signature/lib/enum_sig.py' as '%USERPROFILE%\\Code\\myproj\\shibokensupport\\signature\\lib\\enum_sig.py'
Tracing 'shibokensupport/signature/lib/pyi_generator.py' as '%USERPROFILE%\\Code\\myproj\\shibokensupport\\signature\\lib\\pyi_generator.py'
Tracing 'config.py' as '%USERPROFILE%\\Code\\myproj\\config.py'
Tracing 'config-3.py' as '%USERPROFILE%\\Code\\myproj\\config-3.py'

Specifying source to myproj/ in my pyproject.toml will result in the debug saying "Not Tracing", but coverage shouldn't be looking for non-existent files in the first place. This is clearly a bug.

debug_log.txt

[adam-grant-hendry]

I'm not sure how to help without a way to reproduce this

System

OS: Windows 10
Python: 3.8.10
VSCode: 1.67.2
coverage: 6.4 (with toml extra)

To reproduce,

Minimum Reproducible Example (MRE)

1. Create and activate a virtual environment

   PS> py -m venv .venv
   PS> .venv/Scripts/Activate.ps1

   and ensure vscode sees the Python Interpreter (use Ctrl+Shift+p and set).

2. Create a new poetry project

   PS> poetry init

3. Install pytest, pytest-cov, pytest-env, pytest-qt, pyqt5, and qtpy

   PS> poetry add pytest pytest-cov pytest-qt pyqt5 qtpy

4. Update the pyproject.toml and then run poetry install to make a local (editable) install

   [tool.poetry]
   name = "coverage-issue-1392"
   version = "0.1.0"
   description = ""
   authors = ["Hendry, Adam <adam.grant.hendry@gmail.com>"]
   packages = [
       {include = "myproj"},
       {include = "tests"}
   ]
   
   [tool.poetry.dependencies]
   PyQt5 = "^5.15.6"
   pytest = "^7.1.2"
   pytest-cov = "^3.0.0"
   pytest-env = "^0.6.2"
   pytest-qt = "^4.0.2"
   python = "3.8"
   QtPy = "^2.1.0"
   
   [tool.poetry.dev-dependencies]
   
   [build-system]
   requires = ["poetry-core>=1.0.0"]
   build-backend = "poetry.core.masonry.api"
   
   [tool.pytest.ini_options]
   minversion = "7.0"
   addopts = """\
   --last-failed --last-failed-no-failures all \
   -p no:faulthandler \
   --import-mode=importlib \
   --cov \
   --cov-report term-missing \
   --cov-report html \
   --cov-report xml \
   """
   testpaths = [
       "tests",
   ]
   qt_api = "pyqt5"
   env = [
       "D:COVERAGE_DEBUG=trace",
       "D:COVERAGE_DEBUG_FILE=debug_log.txt"
   ]

5. Create a dummy Qt project

   myproj/main.py

   from qtpy import QtWidgets
   
   
   class MainApp(QtWidgets.QMainWindow):
       """The main window."""
   
   if __name__ == '__main__':
       app = QtWidgets.QApplication([])
       window = MainApp()
       window.show()
       app.exec_()

6. Create a dummy test

   test/test_window.py

   import pytest
   from myproj.main import MainApp
   
   
   @pytest.fixture(name='app', scope='function')
   def fixture_app(qtbot):
       window = MainApp()
       window.show()
   
       qtbot.addWidget(window)
   
       yield window
   
   def test_window_appears(app) -> None:
       assert app.isVisible()

7. Run pytest

   PS> pytest

8. Add pyside6 and remove pyqt5

      PS> poetry add pyside6
      PS> poetry remove pyqt5

   NOTE: Before doing this, be sure to set python exactly to 3.8 in the pyproject.toml:

   python = "3.8"

9. Change qt_api in pyproject.toml:

   [tool.pytest.ini_options]
   qt_api = "pyside6"

10. Update project with new toml

    PS> poetry update

11. Re-run pytest

    PS> pytest

This results in the error messages mentioned in this issue:

%userprofile%\code\coverage-issue-1392\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse '%userprofile%\Code\coverage-issue-1392\pyscript': No source for code: '%userprofile%\Code\coverage-issue-1392\pyscript'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
%userprofile%\code\coverage-issue-1392\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse '%userprofile%\Code\coverage-issue-1392\shibokensupport\__init__.py': No source for 
code: '%userprofile%\Code\coverage-issue-1392\shibokensupport\__init__.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
%userprofile%\code\coverage-issue-1392\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse '%userprofile%\Code\coverage-issue-1392\shibokensupport\feature.py': No source for code: '%userprofile%\Code\coverage-issue-1392\shibokensupport\feature.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
%userprofile%\code\coverage-issue-1392\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\__init__.py': No source for code: '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\__init__.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
%userprofile%\code\coverage-issue-1392\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\errorhandler.py': 
No source for code: '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\errorhandler.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
%userprofile%\code\coverage-issue-1392\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\importhandler.py': No source for code: '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\importhandler.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
%userprofile%\code\coverage-issue-1392\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\layout.py': No source for code: '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\layout.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
%userprofile%\code\coverage-issue-1392\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\lib\__init__.py': 
No source for code: '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\lib\__init__.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
%userprofile%\code\coverage-issue-1392\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\lib\enum_sig.py': 
No source for code: '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\lib\enum_sig.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
%userprofile%\code\coverage-issue-1392\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\lib\pyi_generator.py': No source for code: '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\lib\pyi_generator.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
%userprofile%\code\coverage-issue-1392\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\lib\tool.py': No source for code: '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\lib\tool.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
%userprofile%\code\coverage-issue-1392\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\loader.py': No source for code: '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\loader.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
%userprofile%\code\coverage-issue-1392\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\mapping.py': No source for code: '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\mapping.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
%userprofile%\code\coverage-issue-1392\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\parser.py': No source for code: '%userprofile%\Code\coverage-issue-1392\shibokensupport\signature\parser.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
%userprofile%\code\coverage-issue-1392\.venv\lib\site-packages\coverage\report.py:87: CoverageWarning: Couldn't parse '%userprofile%\Code\coverage-issue-1392\signature_bootstrap.py': No source for code: '%userprofile%\Code\coverage-issue-1392\signature_bootstrap.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")

[nedbat]

This seems to be due to pyside using "shiboken6", which imports files from a zip file, reporting file names that don't exist on the file system. I don't see how coverage.py could know that they shouldn't be attempted.

You can tell reporting to omit those files using settings like this in pyproject.toml:

[tool.coverage.report]
omit = [
    "shibokensupport/*",
    "pyscript",
    "signature_bootstrap.py",
]

Or you can suppress that warning entirely with:

[tool.coverage.run]
disable_warnings = ["couldnt-parse"]

Either way, you will not be bothered with those messages.

[nedbat]

Feel free to reopen this if you have an idea about how coverage.py could do something more here.