python --help output is too long

[merwok]

BPO	46142
Nosy	@warsaw, @vstinner, @merwok, @serhiy-storchaka, @eryksun
PRs	gh-90300: split --help output into separate options #30331

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Linked PRs:

• gh-90300: Reformat the Python CLI help output #93415

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-12-20.23:43:15.950>
labels = ['interpreter-core', 'type-feature', '3.11']
title = 'python --help output is too long'
updated_at = <Date 2022-01-12.01:45:18.299>
user = 'https://github.com/merwok'

bugs.python.org fields:

activity = <Date 2022-01-12.01:45:18.299>
actor = 'eric.araujo'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2021-12-20.23:43:15.950>
creator = 'eric.araujo'
dependencies = []
files = []
hgrepos = []
issue_num = 46142
keywords = ['patch']
message_count = 15.0
messages = ['408981', '408982', '409449', '409450', '409470', '409483', '409738', '409890', '409928', '410052', '410203', '410207', '410238', '410239', '410378']
nosy_count = 5.0
nosy_names = ['barry', 'vstinner', 'eric.araujo', 'serhiy.storchaka', 'eryksun']
pr_nums = ['30331']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue46142'
versions = ['Python 3.11']

Linked PRs

• gh-90300: Reformat the Python CLI help output #93415
• [3.12] gh-90300: Reformat the Python CLI help output (GH-93415) #115847
• gh-90300: Improve the Python CLI help output #115853
• [3.11] gh-90300: Reformat the Python CLI help output (GH-93415) (GH-115847) #116418
• gh-90300: Fix cmdline.rst #116721
• [3.12] gh-90300: Fix cmdline.rst (GH-116721) #116724
• [3.11] gh-90300: Fix cmdline.rst (GH-116721) #116725
• gh-90300: Sort the -X options and some envvars in the Python CLI help #116739
• gh-90300: Document equivalent -X options for envvars in the Python CLI help #116756
• gh-90300: Fix undocumented envvars in the Python CLI help #116765
• [3.12] gh-90300: Sort the -X options and some envvars in the Python CLI help (GH-116739) #116766
• [3.11] gh-90300: Sort the -X options and some envvars in the Python CLI help (GH-116739) (GH-116766) #116779
• [3.12] gh-90300: Document equivalent -X options for envvars in the Python CLI help (GH-116756) #116786
• [3.11] gh-90300: Document equivalent -X options for envvars in the Python CLI help (GH-116756) (GH-116786) #116796
• [3.12] gh-90300: Fix undocumented envvars in the Python CLI help (GH-116765) #116797
• [3.11] gh-90300: Fix undocumented envvars in the Python CLI help (GH-116765) (GH-116797) #116803
• [3.12] gh-90300: Improve the Python CLI help output (GH-115853) #117022
• [3.11] gh-90300: Improve the Python CLI help output (GH-115853) (GH-117022) #117034
• [3.12] gh-90300: Remove reference to PYTHON_FROZEN_MODULES in Python CLI help #117035

[merwok]

From Serhiy in https://mail.python.org/archives/list/python-dev@python.org/thread/QUUBM7DGSXYWBOLZNWOSCQUDALWJIYZF/ :

The output of "python -h" is 104 lines long now. It was only 51 lines in
3.6. 35% of it is about the -X option, and 30% about environment
variables. Also some lines in the -X option description are too long
(102 columns). Both topics are "advanced" and mostly interested for
debugging. I suggest to move them out of the main help output.  […]

Guido:

-X opt : implementation-specific option; use -X help to list options.

We could also see if we can put the help text for each of the supported -X
flags in the table defining these flags […]

GvR again:

--help-env […] prints info about env vars (new flag)

I would enjoy trying to make a patch around next week!

[merwok]

Forgot to quote -X help to print help about X options.

[merwok]

Do people think the man page should be shortened too?

[serhiy-storchaka]

I think that the man page should contain all details related to the CLI (and may be even some examples).

[merwok]

Question about the implementation:
I’ve found the parsing of command-line params in Python/initconfig.c and Python/preconfig.c.
Help is handled in initconfig, X options in preconfig.

A) could add a value to the right struct for "-X help", set it in preconfig, handle it in initconfig (by printing X options help and exiting, maybe reusing the existing print help function or writing a very similar one next to it)
B) or detect the option in preconfig, print and exit right there

Which option is best?

[merwok]

It seems that preconfig is only for some options that change fundamental behaviour (isolated mode, default encoding), so I should be able to detect and handle '-X help' in initconfig.

[vstinner]

initconfig.c parses all -X options. preconfig.c also checks for a few specific -X options which are also checked in initconfig.c.

My notes on parsing command line options:
https://pythondev.readthedocs.io/pyconfig.html#add-a-new-command-line-option

[merwok]

Victor said on github that he would prefer --help-xoptions to -X help. It would align with --help and --help-env, but not with other -X abc options. I think that --help-X may be better, as it uses the same form (X) rather than the non-word xoptions.

What do people think?

[vstinner]

"xoptions" name comes from sys._xoptions (Python) and PyConfig.xoptions (C). Oh, sys._xoptions is a private API... but it's mentioned in the -X documentation:
https://docs.python.org/3/using/cmdline.html#cmdoption-X