Directory src/scriv used to be processed, now: command not found

[nedbat]

Bug description

In scriv, I was using src/scriv as an argument to pylint:

https://github.com/nedbat/scriv/blob/main/tox.ini#L57

With pylint==3.1.0, this worked, and violations were found in files like src/scriv/config.py.

Updating to pylint==3.2.2, it now seems to be running the scriv command, or one of its files that implements the command?:

Command used

pylint src/scriv tests docs setup.py

Pylint output

quality: commands[3]> pylint src/scriv tests docs setup.py
Usage: scriv [OPTIONS] COMMAND [ARGS]...
Try 'scriv --help' for help.

Error: No such command 'src/scriv'.

Expected behavior

I expect the code in src/scriv to be linted.

Pylint version

pylint 3.2.2
astroid 3.2.2
Python 3.8.19 (default, Mar 21 2024, 07:16:56)
[Clang 15.0.0 (clang-1500.3.9.4)]

OS / Environment

Mac 14.5

[jacobtylerwalls]

I'm eager to reproduce this if I can. I cloned scriv and ran python -m tox -e quality and it passed, without any output suggesting the scriv command ran.

I ran without tox as pylint src/scriv tests docs setup.py after installing pylint_pytest, and with pytest 7.4.2, and the files linted, although they threw lots of errors like:

************* Module scriv.util
src/scriv/util.py:12:0: E0401: Unable to import 'click_log' (import-error)
************* Module scriv.cli
src/scriv/cli.py:6:0: E0401: Unable to import 'click_log' (import-error)
************* Module scriv.linkcheck
src/scriv/linkcheck.py:7:0: E0401: Unable to import 'markdown_it' (import-error)
...
************* Module tests.test_ghrel
tests/test_ghrel.py:1:0: F6401: pylint-pytest plugin cannot enumerate and collect pytest fixtures. Please run `pytest --fixtures --collect-only tests/test_ghrel.py` and resolve any potential syntax error or package dependency issues. stdout: . stderr: /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/_pytest/config/__init__.py:324: PluggyTeardownRaisedWarning: A plugin raised an exception during an old-style hookwrapper teardown.
Plugin: helpconfig, Hook: pytest_cmdline_parse
ConftestImportFailure: ModuleNotFoundError: No module named 'responses' (from /Users/jwalls/scriv/tests/conftest.py)
For more information see https://pluggy.readthedocs.io/en/stable/api_reference.html#pluggy.PluggyTeardownRaisedWarning
  config = pluginmanager.hook.pytest_cmdline_parse(
ImportError while loading conftest '/Users/jwalls/scriv/tests/conftest.py'.
tests/conftest.py:10: in <module>
    import responses
E   ModuleNotFoundError: No module named 'responses'
. (cannot-enumerate-pytest-fixtures)

Once I can reproduce, I'm happy to bisect the pylint or astroid changes that could be to blame. I wonder if knowing your installed version of pytest would help. Any other tips for reproducing would be appreciated. Thanks.

[nedbat]

Thanks. I don't see how pytest is involved though.

The main branch of scriv has pylint==3.1.0, and will pass. You can checkout the nedbat/make-upgrade branch, which has pylint==3.2.2, and should fail as shown.

[jacobtylerwalls]

Thanks, I can reproduce now, but also on pylint 3.1 and 3.0. I added a pylint --version to the quality command. I must be missing something else.

quality: commands[0]> pylint --version
pylint 3.0.0
astroid 3.0.0
Python 3.8.10 (v3.8.10:3d8993a744, May  3 2021, 09:09:08) 
[Clang 12.0.5 (clang-1205.0.22.9)]
quality: commands[1]> pylint src/scriv tests docs setup.py
Usage: scriv [OPTIONS] COMMAND [ARGS]...
Try 'scriv --help' for help.

Error: No such command 'src/scriv'.

[nedbat]

Great that you can reproduce it. What did you mean by "I must be missing something else"? Do you mean you don't understand what's causing the behavior yet?

[jacobtylerwalls]

Sorry, I meant that I could reproduce on pylint 3.0 and 3.1, but you report that it was working fine on 3.1, so I'm not sure I've reproduced your specific problem.

[Pierre-Sassoulas]

I've been able to reproduce with:

git clone git@github.com:nedbat/scriv.git
cd scriv
git switch nedbat/make-upgrade
python3 -m venv venv
source venv/bin/activate
pip3 install -e .
pip3 install tox
python -m tox -e quality
git switch main
python -m tox -e quality

(Python 3.10.12, pylint installed/used through tox)

The upgrade for pytest is:

-pytest==8.2.0
+pytest==8.2.1

So probably not linked to pylint-dev/pylint#9576

[Pierre-Sassoulas]

After further investigation and with the following diff:

diff --git a/requirements/quality.txt b/requirements/quality.txt
index 7fdf51c..3ca3d56 100644
--- a/requirements/quality.txt
+++ b/requirements/quality.txt
@@ -8,8 +8,6 @@ alabaster==0.7.13
     # via
     #   -r requirements/doc.txt
     #   sphinx
-astroid==3.2.2
-    # via pylint
 attrs==23.2.0
     # via
     #   -r requirements/doc.txt
@@ -195,7 +193,7 @@ pygments==2.18.0
     #   readme-renderer
     #   rich
     #   sphinx
-pylint==3.2.2
+pylint==2.17.7
     # via
     #   -r requirements/quality.in
     #   pylint-pytest

or

diff --git a/requirements/quality.txt b/requirements/quality.txt
index 7fdf51c..3ca3d56 100644
--- a/requirements/quality.txt
+++ b/requirements/quality.txt
@@ -8,8 +8,6 @@ alabaster==0.7.13
     # via
     #   -r requirements/doc.txt
     #   sphinx
-astroid==3.2.2
-    # via pylint
 attrs==23.2.0
     # via
     #   -r requirements/doc.txt
@@ -195,7 +193,7 @@ pygments==2.18.0
     #   readme-renderer
     #   rich
     #   sphinx
-pylint==3.2.2
+pylint==3.1.1
     # via
     #   -r requirements/quality.in
     #   pylint-pytest

I still have the error, so I suppose the problem come from something else than pylint.

[Pierre-Sassoulas]

By patch adding each dependencies from requirements/quality.txt one by one and bisecting with python -m tox -e quality I blame the pytest upgrade from 8.2.0 to 8.2.1.

[Pierre-Sassoulas]

Thank you for opening the issue though @nedbat ! :)

[nedbat]

What am I missing? How is pytest related to pylint?

[Pierre-Sassoulas]

We also have an issue with pytest's upgrade that I don't understand (I'm bisecting a lot without understanding anything these days 😄) in pylint-dev/pylint#9576 (pytest-dev/pytest#11138 (comment)). So I'm not sure how, but you have a similar issue and the pytest upgrade is what's making your script start failing . Created https://github.com/Pierre-Sassoulas/scriv/tree/pytest-upgrade-to-blame, which reproduce the issue. Here's the log if it makes the way I bisected the issue clearer:
scriv.log

[jacobtylerwalls]

Your repo has the pylint-pytest plugin installed. I don't know or use that plugin, but my assumption is that it uses runtime features of pytest.

[nedbat]

OK, if I remove pylint-pytest, then pylint runs, though it complains about the names of fixtures in the tests, which is the problem the plugin fixes.

[stdedos]

Stop-gap has been deployed:

• https://github.com/pylint-dev/pylint-pytest/releases/tag/v1.1.8
• https://github.com/pylint-dev/pylint-pytest/actions/runs/9204631048
• https://pypi.org/project/pylint-pytest/

[nedbat]

Thanks. The changelog has no entry for it.

Should we write an issue against pytest? I think they would be surprised to hear that pytest==8.2.0 works and 8.2.1 fails.

[nedbat]

Sorry, I'm confused: I don't see a commit on the main branch. And for some reason pip-tools won't pull in version 1.1.8.

[stdedos]

Sorry, I'm confused: I don't see a commit on the main branch

Main branch has been "for a long time" on https://github.com/pylint-dev/pylint-pytest/releases/tag/v2.0.0a0, and therefore not suitable for this fix 😅 (my wish is that v2 will support the current-at-that-time pytest v8 "no matter what").

The branch you are looking for is https://github.com/pylint-dev/pylint-pytest/tree/maint/1.1.x. (I am not planning to make this a habit, but a maintenance branch was "just the thing" for this situation, and the current state)

[stdedos]

And for some reason pip-tools won't pull in version 1.1.8

That is super-weird though. On a new venv:

ua@ha [2024-05-2313:48:47]:/tmp/stdedos/t$ virtualenv .venv
created virtual environment CPython3.8.10.final.0-64 in 272ms
  creator CPython3Posix(dest=/tmp/stdedos/t/.venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/stdedos/.local/share/virtualenv)
    added seed packages: pip==24.0, setuptools==69.5.1, wheel==0.43.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
ua@ha [2024-05-2313:48:54]:/tmp/stdedos/t$ . .venv/bin/ac
activate          activate.csh      activate.fish     activate.nu       activate.ps1      activate_this.py  
ua@ha [2024-05-2313:48:54]:/tmp/stdedos/t$ . .venv/bin/activate
(.venv) ua@h [2024-05-2313:49:05]:/tmp/stdedos/t$ pip install pylint-pytest
Collecting pylint-pytest
  Downloading pylint_pytest-1.1.8-py3-none-any.whl.metadata (6.0 kB)
Collecting pylint>=2 (from pylint-pytest)
  Using cached pylint-3.2.2-py3-none-any.whl.metadata (12 kB)
Collecting pytest<=8.2.0,>=4.6 (from pylint-pytest)
  Using cached pytest-8.2.0-py3-none-any.whl.metadata (7.5 kB)
Collecting platformdirs>=2.2.0 (from pylint>=2->pylint-pytest)
  Using cached platformdirs-4.2.2-py3-none-any.whl.metadata (11 kB)
Collecting astroid<=3.3.0-dev0,>=3.2.2 (from pylint>=2->pylint-pytest)
  Using cached astroid-3.2.2-py3-none-any.whl.metadata (4.5 kB)
Collecting isort!=5.13.0,<6,>=4.2.5 (from pylint>=2->pylint-pytest)
  Using cached isort-5.13.2-py3-none-any.whl.metadata (12 kB)
Collecting mccabe<0.8,>=0.6 (from pylint>=2->pylint-pytest)
  Using cached mccabe-0.7.0-py2.py3-none-any.whl.metadata (5.0 kB)
Collecting tomlkit>=0.10.1 (from pylint>=2->pylint-pytest)
  Downloading tomlkit-0.12.5-py3-none-any.whl.metadata (2.7 kB)
Collecting typing-extensions>=3.10.0 (from pylint>=2->pylint-pytest)
  Using cached typing_extensions-4.11.0-py3-none-any.whl.metadata (3.0 kB)
Collecting dill>=0.2 (from pylint>=2->pylint-pytest)
  Using cached dill-0.3.8-py3-none-any.whl.metadata (10 kB)
Collecting tomli>=1.1.0 (from pylint>=2->pylint-pytest)
  Using cached tomli-2.0.1-py3-none-any.whl.metadata (8.9 kB)
Collecting iniconfig (from pytest<=8.2.0,>=4.6->pylint-pytest)
  Using cached iniconfig-2.0.0-py3-none-any.whl.metadata (2.6 kB)
Collecting packaging (from pytest<=8.2.0,>=4.6->pylint-pytest)
  Using cached packaging-24.0-py3-none-any.whl.metadata (3.2 kB)
Collecting pluggy<2.0,>=1.5 (from pytest<=8.2.0,>=4.6->pylint-pytest)
  Using cached pluggy-1.5.0-py3-none-any.whl.metadata (4.8 kB)
Collecting exceptiongroup>=1.0.0rc8 (from pytest<=8.2.0,>=4.6->pylint-pytest)
  Using cached exceptiongroup-1.2.1-py3-none-any.whl.metadata (6.6 kB)
Downloading pylint_pytest-1.1.8-py3-none-any.whl (10 kB)
Using cached pylint-3.2.2-py3-none-any.whl (519 kB)
Using cached pytest-8.2.0-py3-none-any.whl (339 kB)
Using cached astroid-3.2.2-py3-none-any.whl (276 kB)
Using cached dill-0.3.8-py3-none-any.whl (116 kB)
Using cached exceptiongroup-1.2.1-py3-none-any.whl (16 kB)
Using cached isort-5.13.2-py3-none-any.whl (92 kB)
Using cached mccabe-0.7.0-py2.py3-none-any.whl (7.3 kB)
Using cached platformdirs-4.2.2-py3-none-any.whl (18 kB)
Using cached pluggy-1.5.0-py3-none-any.whl (20 kB)
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Downloading tomlkit-0.12.5-py3-none-any.whl (37 kB)
Using cached typing_extensions-4.11.0-py3-none-any.whl (34 kB)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Using cached packaging-24.0-py3-none-any.whl (53 kB)
Installing collected packages: typing-extensions, tomlkit, tomli, pluggy, platformdirs, packaging, mccabe, isort, iniconfig, exceptiongroup, dill, pytest, astroid, pylint, pylint-pytest
Successfully installed astroid-3.2.2 dill-0.3.8 exceptiongroup-1.2.1 iniconfig-2.0.0 isort-5.13.2 mccabe-0.7.0 packaging-24.0 platformdirs-4.2.2 pluggy-1.5.0 pylint-3.2.2 pylint-pytest-1.1.8 pytest-8.2.0 tomli-2.0.1 tomlkit-0.12.5 typing-extensions-4.11.0
ua@ha [2024-05-2313:49:12]:/tmp/stdedos/t$

"Anything you can think of" that could be the problem?

[nedbat]

I solved the pinning problem: I have a multi-phase run of pip-tools. Adding pylint-pytest into the same phase as pytest solved the problem.

[stdedos]

Oh yeah, I see the problem (nedbat/scriv@4fdfbce).
I didn't fully dug that out, but maybe it's a pip-tools feature? or maybe all test[x]-co-dependencies need to be "resolved" together? idk

Since the stop-gap is "good enough" for now, I think I'll go ahead and close this.

Do feel comfortable continuing discussion here, and/or pass by #68 and give debugging hints 🙏

[pzelnip]

Is there a plan to fix pylint-pytest to be able to be compatible with pytest 8.2.1? As it stands my dependency management tool keeps trying to update pytest (with pylint-pytest downgraded to 1.1.7), then it sees that there's a new version of pylint-pytest, upgrades it and downgrades pytest back to 8.2.0, then repeats the cycle. 😞

[stdedos]

Yeah - I think that renovatebot should have updated pylint-pytest to 1.1.8, which should've blocked you from updating to pytest 8.2.1. I am not sure why renovatebot cannot handle that 😕

If you open a discussion at https://github.com/renovatebot/renovate/discussions/, feel free to mention me

Is there a plan to fix pylint-pytest to be able to be compatible with pytest 8.2.1?

Yes, there are plans: https://github.com/pylint-dev/pylint-pytest/milestone/2
Right now, though, I don't have enough time to debug #68 - which troubles me very much from releasing v2

[pzelnip]

Opened renovatebot/renovate#29469 on Renovate's repo. Thanks!