TypeError: <lambda>() missing 1 required positional argument: 'scope_type'

[TheSuperiorStanislav]

Bug description

Pylint crashes with the following error

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/__init__.py", line 22, in run_pylint
    PylintRun(argv or sys.argv[1:])
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/lint/run.py", line 358, in __init__
    linter.check(args)
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 1077, in check
    check_parallel(
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/lint/parallel.py", line 157, in check_parallel
    for (
  File "/home/stanislav/.pyenv/versions/3.9.8/lib/python3.9/multiprocessing/pool.py", line 870, in next
    raise value
TypeError: <lambda>() missing 1 required positional argument: 'scope_type'

Configuration

[MASTER]
ignore=migrations
jobs=0
suggestion-mode=yes

# Enable the plugins
load-plugins=pylint_django,
             pylint_pytest,
             pylint_quotes,
             pylint.extensions.docstyle,
             pylint.extensions.docparams,

accept-no-param-doc=no
default-docstring-type=google

ignored-modules=invoke

string-quote=double
triple-quote=double
docstring-quote=double

# Settings not configured because of
# `config/__init__.py` that imports celery app, that configure settings
# by itself
# django-settings-module=config.settings.local

[MESSAGES CONTROL]
disable:no-self-use,
        too-few-public-methods,
        missing-return-doc,
        missing-return-type-doc,
        missing-any-param-doc,
        missing-param-doc,
        missing-type-doc,
        missing-module-docstring,
        too-many-ancestors,
        import-outside-toplevel,
        unsupported-binary-operation,
        keyword-arg-before-vararg,
        logging-fstring-interpolation,
        duplicate-code,
        bad-docstring-quotes,
        fixme

[FORMAT]
# Maximum number of characters on a single line.
max-line-length=79
ignore-long-lines=<?https?://

[REPORTS]
score=yes
# Set the output format. Available formats are text, parseable, colorized, msvs
# (visual studio) and html
output-format=parseable

# Template used to display messages. This is a python new-style format string
# used to format the message information. See doc for all details.
msg-template='{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}'

# Include message's id in output
#include-ids=yes

# Tells whether to display a full report or only the messages
reports=n
ignored-argument-names=args|kwargs

# Regular expression which should only match functions or classes name which do
# not require a docstring
no-docstring-rgx=^_|__init__|Meta|__str__|unicode

Command used

pylint apps libs provision conftest.py tasks.py

Pylint output

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/__init__.py", line 22, in run_pylint
    PylintRun(argv or sys.argv[1:])
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/lint/run.py", line 358, in __init__
    linter.check(args)
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 1077, in check
    check_parallel(
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/lint/parallel.py", line 157, in check_parallel
    for (
  File "/home/stanislav/.pyenv/versions/3.9.8/lib/python3.9/multiprocessing/pool.py", line 870, in next
    raise value
TypeError: <lambda>() missing 1 required positional argument: 'scope_type'

Expected behavior

It does not crash

Pylint version

pylint 2.13.0
astroid 2.11.1
Python 3.9.8 (main, Jan 10 2022, 12:02:40) 
[GCC 11.1.0]

OS / Environment

 Static hostname: stanislav-pc
       Icon name: computer-desktop
         Chassis: desktop 🖥
      Machine ID: 2c95b0fa18c24bb2ba52190b5b31f03c
         Boot ID: a586d9d94d0e4bbf884479a8ebf4860e
Operating System: Manjaro Linux                   
          Kernel: Linux 5.10.105-1-MANJARO
    Architecture: x86-64

Additional dependencies

Package               Version
--------------------- --------
amqp                  5.1.0
argcomplete           1.12.3
arrow                 1.2.2
asgiref               3.5.0
astroid               2.11.1
asttokens             2.0.5
async-timeout         4.0.2
attrs                 21.4.0
backcall              0.2.0
billiard              3.6.4.0
boto3                 1.21.26
botocore              1.24.26
celery                5.2.3
cffi                  1.15.0
cfgv                  3.3.1
click                 8.0.4
click-didyoumean      0.3.0
click-plugins         1.1.1
click-repl            0.2.0
colorama              0.4.4
commitizen            2.21.2
commonmark            0.9.1
cryptography          36.0.2
decli                 0.5.2
decorator             5.1.1
defusedxml            0.7.1
Deprecated            1.2.13
diff-match-patch      20200713
dill                  0.3.4
distlib               0.3.4
Django                3.2.12
django-autoslug       1.9.8
django-celery-beat    2.2.1
django-cities-light   3.9.0
django-cors-headers   3.11.0
django-debug-toolbar  3.2.4
django-dirtyfields    1.8.1
django-extensions     3.1.5
django-filter         21.1
django-health-check   3.16.5
django-import-export  2.7.1
django-object-actions 4.0.0
django-picklefield    3.0.1
django-redis          5.2.0
django-rest-knox      4.2.0
django-s3direct       1.1.5
django-storages       1.12.3
django-taggit         2.1.0
django-timezone-field 4.2.3
djangorestframework   3.13.1
drf-spectacular       0.21.2
et-xmlfile            1.1.0
execnet               1.9.0
executing             0.8.3
factory-boy           3.2.1
Faker                 13.3.3
filelock              3.6.0
flake8                4.0.1
flake8-bugbear        22.3.23
flake8-commas         2.1.0
flake8-plugin-utils   1.3.2
flake8-pytest-style   1.6.0
identify              2.4.12
inflection            0.5.1
iniconfig             1.1.1
invoke                1.7.0
ipdb                  0.13.9
ipython               8.1.1
isort                 5.10.1
jedi                  0.18.1
Jinja2                3.0.3
jmespath              1.0.0
jsonschema            4.4.0
kombu                 5.2.4
lazy-object-proxy     1.7.1
MarkupPy              1.14
MarkupSafe            2.1.1
matplotlib-inline     0.1.3
mccabe                0.6.1
mistune               2.0.2
nodeenv               1.6.0
odfpy                 1.4.1
openpyxl              3.0.9
packaging             21.3
parso                 0.8.3
pep517                0.12.0
pexpect               4.8.0
pickleshare           0.7.5
Pillow                9.0.1
pip                   22.0.4
pip-tools             6.5.1
platformdirs          2.5.1
pluggy                1.0.0
pre-commit            2.17.0
progressbar2          4.0.0
prompt-toolkit        3.0.28
psycopg2-binary       2.9.3
ptyprocess            0.7.0
pure-eval             0.2.2
py                    1.11.0
pycodestyle           2.8.0
pycparser             2.21
pyflakes              2.4.0
Pygments              2.11.2
pylint                2.13.0
pylint-django         2.5.2
pylint-plugin-utils   0.7
pylint-pytest         1.1.2
pylint-quotes         0.2.3
pyparsing             3.0.7
pyrsistent            0.18.1
pytest                7.1.1
pytest-django         4.5.2
pytest-forked         1.4.0
pytest-sugar          0.9.4
pytest-xdist          2.5.0
python-crontab        2.6.0
python-dateutil       2.8.2
python-utils          3.1.0
pytz                  2022.1
pyuwsgi               2.0.20
PyYAML                6.0
questionary           1.10.0
redis                 4.2.0
rich                  12.0.1
s3transfer            0.5.2
setuptools            59.6.0
six                   1.16.0
sqlparse              0.4.2
stack-data            0.2.0
tablib                3.2.0
termcolor             1.1.0
toml                  0.10.2
tomli                 2.0.1
tomlkit               0.10.0
traitlets             5.1.1
typing_extensions     4.1.1
Unidecode             1.3.4
uritemplate           4.1.1
urllib3               1.26.9
uwsgitop              0.11
vine                  5.0.0
virtualenv            20.13.4
wcwidth               0.2.5
Werkzeug              2.0.3
wheel                 0.37.1
wrapt                 1.14.0
xlrd                  2.0.1
xlwt                  1.3.0

[Pierre-Sassoulas]

Thank you for opening the issue. Could you provide the code causing the crash or maybe the full stacktrace with the original error, please ?

[TheSuperiorStanislav]

This is the only thing pylint logs. And pylint pointing to backend/apps/__init__.py but it's empty.

Exception on node <Module.apps l.0 at 0x7f7ce0b9ad90> in file '/home/stanislav/PycharmProjects/backend/apps/__init__.py'
Traceback (most recent call last):
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/utils/ast_walker.py", line 78, in walk
    callback(astroid)
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint_django/augmentations/__init__.py", line 770, in wrap_func
    return with_method(orig_method, *args, **kwargs)
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint_django/augmentations/__init__.py", line 339, in ignore_import_warnings_for_related_fields
    consumer._atomic = ScopeConsumer(new_things, consumer.consumed, consumer.scope_type)  # pylint: disable=W0212
TypeError: <lambda>() missing 1 required positional argument: 'scope_type'
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/stanislav/.pyenv/versions/3.9.8/lib/python3.9/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/lint/parallel.py", line 80, in _worker_check_single_file
    _worker_linter.check_single_file_item(file_item)
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 1100, in check_single_file_item
    self._check_file(self.get_ast, check_astroid_module, file)
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 1146, in _check_file
    check_astroid_module(ast_node)
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 1298, in check_astroid_module
    retval = self._check_astroid_module(
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 1345, in _check_astroid_module
    walker.walk(node)
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/utils/ast_walker.py", line 78, in walk
    callback(astroid)
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint_django/augmentations/__init__.py", line 770, in wrap_func
    return with_method(orig_method, *args, **kwargs)
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint_django/augmentations/__init__.py", line 339, in ignore_import_warnings_for_related_fields
    consumer._atomic = ScopeConsumer(new_things, consumer.consumed, consumer.scope_type)  # pylint: disable=W0212
TypeError: <lambda>() missing 1 required positional argument: 'scope_type'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/__init__.py", line 22, in run_pylint
    PylintRun(argv or sys.argv[1:])
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/lint/run.py", line 358, in __init__
    linter.check(args)
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 1077, in check
    check_parallel(
  File "/home/stanislav/.pyenv/versions/3.9.8/envs/backend/lib/python3.9/site-packages/pylint/lint/parallel.py", line 157, in check_parallel
    for (
  File "/home/stanislav/.pyenv/versions/3.9.8/lib/python3.9/multiprocessing/pool.py", line 870, in next
    raise value
TypeError: <lambda>() missing 1 required positional argument: 'scope_type'

[DanielNoord]

Could this be an issue in pylint_django? I'm seeing it in the stacktrace.
@carlio has something like this been reported against you previously?

[carlio]

I have not seen this reported, no (yet!).

However I have had a look and I think it's related to the pylint 2.13 release.

In 2.12.2 ScopeConsumer has 3 fields - https://github.com/PyCQA/pylint/blob/v2.12.2/pylint/checkers/variables.py#L537

Whereas in 2.13 it has 4 - https://github.com/PyCQA/pylint/blob/v2.13.0/pylint/checkers/variables.py#L520

And pylint-django uses it once, here - https://github.com/PyCQA/pylint-django/blob/master/pylint_django/augmentations/__init__.py#L339

So I will need to add something to pylint-django to give either 3 or 4 arguments to that constructor, depending on pylint version - not a big deal.

However just wanted to let you know that the change is not backwards compatible between versions and you may run into other reports if any other plugins are using that ScopeConsumer tuple.

[DanielNoord]

Hm, I don't think we consider pylint's own checkers to be external API, but I can see why plugins might want to tie into this particular checker.

@jacobtylerwalls Do you see possibility to change this to become backwards compatible without adding to much overhead/redundancy? Perhaps with default values for the NamedTuple and keeping the order of the arguments like they previously were?

[carlio]

I'm not sure if it needs to be backwards compatible or not. pylint-django is odd because it does a lot of frobbing of pylint internals so changes like this just need to have pylint-django patches made! It's certainly not expected that everything internal doesn't change. (I should turn on the nightly builds against main again to catch these before release)

[Pierre-Sassoulas]

Thanks for the quick reply @carlio. I think reinstating bleeding edge pylint tests in pylint django is the right way to anticipate this. (As we had the discussion in the past : Just do not set the astroid version, we're updating astroid ourselves when we think it's ready for the current pylint 😉)

[leunardo]

I had the same problem today. I am dropping my crash log here, hope it helps :)
pylint-crash-2022-03-25-10.txt