fail/warn on using parallel execution with custom plugins

[stanislavlevin]

According to documentation:
http://pylint.pycqa.org/en/latest/user_guide/run.html#parallel-execution

There are some limitations in running checks in parallel in the current implementation. It is not possible to use custom plugins (i.e. --load-plugins option)...

Actually, it is possible but silently broken.
If this is still by design then Pylint should inform a user about it in such cases.
As for now, I could run:

pylint -j 10 --load-plugins plugin_foo bar.py

without any warning or error.
Unfortunately, linting results are not the same as a single process linting, but Pylint silently pass. So, results are not predictable.

Proposal: emit a warning or better explicitly fail on using parallel execution with custom Pylint plugins, because people usually don't read the documentation while things works.

[PCManticore]

Thanks for opening an issue @stanislavlevin I agree, we should communicate this better. I think we should check that it's still the case, and if so, we should add some runtime warnings to make it more obvious (or maybe raising an error when we detect custom plugins and the parallel mode).

[stanislavlevin]

Original ticket:
https://pagure.io/freeipa/issue/8116

[adamtheturtle]

This is still the case - I just spent some time figuring out why my plugin was not loading and found those docs eventually. A warning would have helped greatly.

cc @PCManticore

[clavedeluna]

I've started work on this and got

        if linter.config.jobs >= 0:
            if self._plugins:
                warnings.warn(
                    "Running pylint in parallel with custom plugins is not currently supported.",
                    UserWarning,
                )
                # sys.exit(32)

but wanted to check if we want both a warning and a crash (sys.exit) or just a warning. With the crash I have to update quite a few tests so I wanted to make sure before investing that time.

[DanielNoord]

Tagging @Pierre-Sassoulas for an opinion on this!

[Pierre-Sassoulas]

I'm not super up to date on this issue but imo if the result would be rubbish and plain wrong we need to crash, if it's still usable with a possible issue and we're not sure about it then we should warn.

[clavedeluna]

Given original statement

Unfortunately, linting results are not the same as a single process linting, but Pylint silently pass. So, results are not predictable.

sounds like we should indeed crash.