Argparse does not identify negative numbers with underscores as a numerical value

[coppinri]

Bug report

Bug description:

Simple example:

$ python test_arguments.py --Value -1_000.0
error: argument -v/--Value: expected one argument

The regular expression to identify negative numbers in the argparse.py module uses the following regular expression:

#argparse.py:1353-1354
        # determines whether an "option" looks like a negative number
        self._negative_number_matcher = _re.compile(r'^-\d+$|^-\d*\.\d+$')

This does not identify negative numerical values with underscores as a numerical value

This could probably be resolved by changing the regex to:

        # determines whether an "option" looks like a negative number
        self._negative_number_matcher = _re.compile(r'^-\d[\d_]*$|^-[\d_]*\.\d+$')

CPython versions tested on:

3.9

Operating systems tested on:

Windows

Linked PRs

• GH-123945: Update regex for parsing negative numbers that contain underscores #123970
• [3.13] GH-123945: Update regex for parsing negative numbers that contain underscores (GH-123970) #124158
• [3.12] GH-123945: Update regex for parsing negative numbers that contain underscores (GH-123970) #124175

[hugovk]

Small reproducer:

import argparse

parser = argparse.ArgumentParser()
parser.add_argument("--int", type=int)
parser.add_argument("--float", type=float)
args = parser.parse_args()
print(args)

Reproducible on main. Positive values will accept underscore, but not negative:

❯ p 1.py --int 1000
Namespace(int=1000, float=None)

❯ p 1.py --int 1_000
Namespace(int=1000, float=None)

❯ p 1.py --int -1000
Namespace(int=-1000, float=None)

❯ p 1.py --int -1_000
usage: 1.py [-h] [--int INT] [--float FLOAT]
1.py: error: argument --int: expected one argument

❯ p 1.py --float 1000.1
Namespace(int=None, float=1000.1)

❯ p 1.py --float 1_000.1
Namespace(int=None, float=1000.1)

❯ p 1.py --float -1000.1
Namespace(int=None, float=-1000.1)

❯ p 1.py --float -1_000.1
usage: 1.py [-h] [--int INT] [--float FLOAT]
1.py: error: argument --float: expected one argument

[savannahostrowski]

I'll take a look at this!

[coppinri]

Thanks for picking this up.

Just a quick note on the proposed regex:

The proposed regex may accept numbers like: -_.001 that may not be inline with the python parser, but I haven't check with the

There is also no provision for underscores in the fractional part of the number.

[serhiy-storchaka]

The last change introduced a regression: -.5 no longer accepted as a negative number.

[serhiy-storchaka]

I am not actually sure that this issue should be classified as a bug. There was no promise that argparse supports all numbers supported by Python (and it does not support numbers in scientific notations like -1e3, or complex numbers like -1j, or different decimal separators like -12,3). Underscores are supported for readability, but humans do not use underscores when they specify negative numbers for argparse (because this does not work). Generated data also usually do not contain underscores in numbers. So this change can help future users of argparse, but does not fix existing issue (for which users need to use workarouds).

[hauntsaninja]

Yeah, I agree, let me close the backports.

[hauntsaninja]

Okay great, looks like Savannah fixed the regression in #124321 , so I think we can call this one closed 🎉

[gpshead]

fwiw that makes sense, supporting parsing these is more "feature-ish". thanks for closing the backport PRs.

[serhiy-storchaka]

I wonder whether we should make the check even more lenient, for example -\.?\d. This would help the user to support all numbers mentioned above (-1e3, -1j, -12,3 and also hexadecimals -0xabc, etc) if they use custom converter. Currently the only workaround is to override private member _negative_number_matcher in the parser constructor and the add_mutually_exclusive_group() method.

[picnixz]

At some point I needed something to accept floats using all kind of notations. So what I did is replace the action by some ast.literal_eval call. Do you think a LiteralEval action would make sense actually? in addition, we could have a type for that action which would then check that the evaluation's result is of correct type (we could also restrict ast.literal_eval so that only truly constant expressions are accepted, and reject arithmetic)

[savannahostrowski]

@serhiy-storchaka Right now, we actually do accept numbers like positive versions of scientific notation and complex numbers...but not negative (we didn't before either). I will open a PR to get us to parity.