ArgumentParser negative number regex too restrictive

[marcusmueller]

Bug report

Background Story

GNU Radio has a python program design tool, the GNU Radio companion. It creates Python programs from visually designed flow graphs.

User expect to be able to define command line parameters. We support that by generating ArgumentParser-based argument parsers.

Since GNU Radio is used a lot with scientific notation, it's very common for users to enter numbers such as 2.4e9 on the command line, or -3e-12.

The latter is what breaks

Problem

parser.add_argument("-p", "--param", type=float)
args = parser.parse_args()

works with

./prog.py -p 3e12
./prog.py -p -42

but doesn't work with

./prog.py -p -3e12

as the latter is misinterpreted, with -3e12 being interpreted as optional argument that's undefined.

Root analysis

argparse.py specifies

cpython/Lib/argparse.py

Lines 1402 to 1403 in 8da9d1b

	# determines whether an "option" looks like a negative number
	self._negative_number_matcher = _re.compile(r'^-\d+$|^-\d*\.\d+$')

That's too restrictive.

^-\d+$|^-\d*\.\d+$|^-\d*\.?\d+e[+-]?\d+$

would be appropriate and actually cover most negative number formats

Your environment

Python 3.11.3 (F37 x86), but also CPython main (v3.12.0b1-321-g8da9d1b163)

Impact

Yeah, it's annoying, because we can't go out and repair deployed code, monkey-patch everyone's parser. This leads to surprising outages.

The proposed alternative regex might be made default, with the option to switch back to the over-restrictive parser if arguments starting with -\d*\.?\d+e are registered (which, honestly, sounds unlikely).

Related issue

#53580 which I found is related, but less solution-oriented seems to have been closed as "WONTFIX" with the slightly vague argument that the change is hard (based on proposing an overly simple regex). Seeing my proposal, hm, not sure I agree.

Alternatives

Alternative Regexes

(that regex I propose might not be optional. If we can actually find the regex that describes negative floating point numbers, that's what I'd propose we use)

Passing of a user-defined negative-number regex

Since monkeypatching does work, at least for the main parser, a colleague brought up the idea of adding a keyword argument to ArgumentParser which would allow for user-defined modification of that string, at zero risk of breaking existing use cases

Linked PRs

• gh-105712: Use appropriate regex to detect negative numbers in arguments #114833

[vadmium]

Same bug as #66862 and #96994

[hugovk]

Closing as duplicate.