With `--jobs 2`, the `open()` call is executed for each opened file

[Pierre-Sassoulas]

With --jobs 2, the open() call is executed for each opened file. E.g. for HomeAssistant I got over 16.000 warnings just for 3 old config values. That should be fixed before we release 2.16.

https://github.com/cdce8p/ha-core/actions/runs/3470026809/jobs/5797737991#step:8:25

pylint: Command line or configuration file:1: UserWarning: Specifying exception names in the overgeneral-exceptions option without module name is deprecated and support for it will be removed in pylint 3.0. Use fully qualified name (maybe 'builtins.BaseException' ?) instead.
pylint: Command line or configuration file:1: UserWarning: Specifying exception names in the overgeneral-exceptions option without module name is deprecated and support for it will be removed in pylint 3.0. Use fully qualified name (maybe 'builtins.Exception' ?) instead.
pylint: Command line or configuration file:1: UserWarning: Specifying exception names in the overgeneral-exceptions option without module name is deprecated and support for it will be removed in pylint 3.0. Use fully qualified name (maybe 'builtins.HomeAssistantError' ?) instead.
pylint: Command line or configuration file:1: UserWarning: Specifying exception names in the overgeneral-exceptions option without module name is deprecated and support for it will be removed in pylint 3.0. Use fully qualified name (maybe 'builtins.BaseException' ?) instead.
pylint: Command line or configuration file:1: UserWarning: Specifying exception names in the overgeneral-exceptions option without module name is deprecated and support for it will be removed in pylint 3.0. Use fully qualified name (maybe 'builtins.Exception' ?) instead.
pylint: Command line or configuration file:1: UserWarning: Specifying exception names in the overgeneral-exceptions option without module name is deprecated and support for it will be removed in pylint 3.0. Use fully qualified name (maybe 'builtins.HomeAssistantError' ?) instead.
pylint: Command line or configuration file:1: UserWarning: Specifying exception names in the overgeneral-exceptions option without module name is deprecated and support for it will be removed in pylint 3.0. Use fully qualified name (maybe 'builtins.BaseException' ?) instead.
pylint: Command line or configuration file:1: UserWarning: Specifying exception names in the overgeneral-exceptions option without module name is deprecated and support for it will be removed in pylint 3.0. Use fully qualified name (maybe 'builtins.Exception' ?) instead.
pylint: Command line or configuration file:1: UserWarning: Specifying exception names in the overgeneral-exceptions option without module name is deprecated and support for it will be removed in pylint 3.0. Use fully qualified name (maybe 'builtins.HomeAssistantError' ?) instead.
pylint: Command line or configuration file:1: UserWarning: Specifying exception names in the overgeneral-exceptions option without module name is deprecated and support for it will be removed in pylint 3.0. Use fully qualified name (maybe 'builtins.BaseException' ?) instead.

Originally posted by @cdce8p in #7497 (comment)

[jacobtylerwalls]

I don't think we will be able to refactor the parallelism to avoid calling open() before 2.16, that's a huge task for 3.0. To fix the blocker, maybe we can add a warnings filter "once" around our own warning ... is that feasible?

[Pierre-Sassoulas]

We did something similar to avoid crash messages spamming by creating a file with a date in the name.

[Pierre-Sassoulas]

I did some research this is indeed more complicated than I though, as no simple filter work work on process pool executor. Here's a simple reproducer:

import warnings
from concurrent.futures import ProcessPoolExecutor


def a_warning():
    warnings.warn("Hello filter my old friend.")


with ProcessPoolExecutor(max_workers=10, initializer=a_warning) as executor:
    warnings.simplefilter("once")
    executor.submit(a_warning)

I'm going to remove the blocker, this is something that is going to happen once and spam you for the number of job you defined, we can probably live with it.

[licui7]

I have this bug even with python v3.10.9 for pylint v2.16.0. This bug is gone with pylint 2.16.1 though.