Migration to argparse from optparse

[Pierre-Sassoulas]

Current problem

optparse is deprecated since python 3.2 and is not developed anymore. Also, the parsing are handled in a strange way with callback that created stupid bug that could probably be avoided by refactoring the code (#3581) We need to migrate to another CLI parser, the most logical being argparse or click, but confuse is also a possibility because it permits to have default option in a configuration file that could be one of the configuration we already handle and make option / configuration consistent. I'm open to other libraries too.

Desired solution

No more optparse used in the code, small and modular handling of CLI argument parsing, no more special case, or minimal.

Additional context

#5152 (comment)
#5259 (comment)

[DanielNoord]

I think argparse makes the most sense as it is the direct successor to optparse.

To do this I think it makes the most sense to do some initial preparation, namely convert all options to a new class (we could use the one I proposed in #5152) and move them all to PyLinter. This avoids the current problem we're having with double settings etc and will probably also be faster (?) as we only need to parse arguments once instead for each individual checker.

[saroad2]

Even though argparse is a built-in library, and a way better solution than optparse, I strongly recommend using Click as your CLI library:

• It makes a shorter and cleaner code
• It has a lot of neat features that makes it so much easy to write CLI with.
• It is a living project that still publish new abilities and features
• It is supported by its developers
• It's widely used in the industry (personally, I use it all the time and in all of my projects)
• It includes testing tools that helps you test your CLI

[Pierre-Sassoulas]

I like click, but my concern is it adds a new dependency. Also argparse is compatible with confuse which would be very nice to help with consistency between options and configuration file enforced in a simple way.

[saroad2]

In my humble opinion, adding a dependency is not that of a problem. In the end it's all matter of "buy what you won't implement", using a library instead of developing something out of your domain. I think that click answers to this definition.

As to confuse, I am not familiar with this library so I don't have a strong opinion about it. In the end it's all matter of pros and cons. From my own experience with both argparse and click, I think click brings much more wholistic solution to the CLI programming process. But again, one an argue otherwise.

[DanielNoord]

I have started work on migration to argparse. I have not previous experience with any of the alternatives mentioned in this issue, but I think trying to depend only on stdlib is a good thing for a static checker such as pylint. It allows everybody to use pylint in combination with other linter or analysers.

One of the things that I think we should change (and I'm changing in my proposal) is that options are no longer registered to each individual checker class but to the PyLinter object. This is preparing for (eventually) supporting sub-directory configuration:
If we create helper functions that can be passed an argparse.Namespace (or look up the correct Namespace) and return the current value of an option, we can create Namespaces for individual directories. This is similar to how we're already using get_global_option. We would just need to enforce the use of these helpers more strictly, but I think that approach might work (and fix a long standing issue with pylint).

[DanielNoord]

@Pierre-Sassoulas I want to ask for input before committing to this:
Do we want to keep requiring the sections in options files? It adds a lot of extra checks in the code and honestly I don't see why we should require it. It's trivial to keep supporting it but I would like to support:

[tool.pylint]
missing-member-hint = true

[tool.pylint."TYPECHECK"]
missing-member-hint = true

[tool.pylint."I-ORDER-MY-SECTIONS-THE-WAY-I-WANT-TO"]
missing-member-hint = true

I do not immediately see what having these sections internally offers to us.