argparse eats initial '=' from short option argument

[netch80]

Bug report

Bug description:

When configured with a short option that requires an argument, and an input commandline has the option argument at the same argv item, argparse deletes initial '=' from it. This behavior is inconsistent with other implementations of the same parsing (at least: getopt, getopt_long from glibc or FreeBSD libc; getopt and optparse from Python standard library).

Actual result: initial '=' is deleted under the described conditions.

Expected result: initial '=' is kept in the collected option value.

Example code:

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-Q', type=str)
args = parser.parse_args()
print(getattr(args, 'Q', None))

Running it:

$ ./t_eq_argparse.py -Q22
22
$ ./t_eq_argparse.py -Q=22
22

With a separate argument, truncating has no place:

$ ./t_eq_argparse.py -Q =22
=22

Compare to getopt module:

$ ./t_eq_getopt.py -Q22
22
$ ./t_eq_getopt.py -Q=22
=22

The same result with libc getopt (Ubuntu 22.04) and optparse module. Test code for getopt and optparse is obvious.

The issue is present with today's Lib/argparse.py from github trunk.

CPython versions tested on:

3.10, CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-124305: argparse no longer eats initial '=' from short option argument #124332

[netch80]

Credits for finding to: discussion at: https://habr.com/ru/companies/timeweb/articles/840200/

[savannahostrowski]

While I don't have the historical context to understand why this behaviour differs between optparse and argparse, I feel like this is expected behaviour. Using = as a delimiter between the option and the value is pretty standard.

Changing the behaviour is also likely to be a hugely breaking change at this point, given that many folks rely on this convention.

[netch80]

Using = as a delimiter between the option and the value is pretty standard.

It is not. It could be in another universe, but this is not how other libraries work now.

Forcing '=' between option name and value, if they are in the same argv item, is the matter of fact for long options because 1) they need an explicit name terminator and 2) they allow optional argument case (and this could have been also done because of opportunity by (1)). But this is not how short options work. They are either without an argument or with mandatory argument, and '=' is not added in the latter case.

You may, of course, make the current behavior nailed and documented... but you'll gather curses from all programmers that face the discrepancy.

given that many folks rely on this convention.

Extremely unlikely, because default expectation is what is average among the whole industry - and this is not how argparse works to date.

[lsap]

Let me post more specific link to the discussion: https://habr.com/en/companies/timeweb/articles/840200/comments/

[serhiy-storchaka]

It is unfortunate that the behavior of argparse differs from that of other parsers in this particular case. There are reasons for this:

• argparse supports long options with a single - prefix, e.g. -foo.
• argparse supports abbreviation of options, and it is enabled by default. So -f and -fo may be abbreviations of long option -foo.

As result, -f may require = if it is an abbreviation of a long option to distinguish the cases when -fo is and abbreviation of -foo and when it is a short option -f with argument o.

It does not mean that we should not try to resolve conflict between argparse and other parsers. I'll look what can be done.

[serhiy-storchaka]

#124332 fixes this issue. The only failed tests are tests added by me in e47ecbd (GH-114180) few months ago. They do not reflect intended behavior, they only check that the current behavior (correct or not) will not change unintentionally. So it is unlikely that this fix will break some user code. This behavior was not documented and not tested.

[serhiy-storchaka]

There is a problem here. How can you pass an option with an arbitrary value?

1. -f "$ARG"
   This does not work if $ARG starts with - (in particularly if it is --). argparse consider it as a different option and does not allow to use as optional argument value. Eating -- would have catastrophic consequences -- argparse does not work with this.

2. -f"$ARG"
   This does not work if $ARG starts with = (this bug) or if it is empty.

3. -f="$ARG"
   This always works, but will stop to work after fixing this bug.

It is difficult to fix case 1. It would require significant rewriting of some code in the core of argparse. It will take time. But until it is fixed, we cannot fix 2 by breaking 3. So, although the fix is ​​ready, it has to wait.