Floating point syntax specification is incorrect

[knotapun]

Documentation

The floating point syntax spec is incorrect, and can be found in /Doc/library/functions.rst:

The syntax rendered currently is the following:

sign        ::=  "+" | "-"
infinity    ::=  "Infinity" | "inf"
nan         ::=  "nan"
digitpart   ::=  `!digit` (["_"] `!digit`)* 
number      ::=  [digitpart] "." digitpart | digitpart ["."]
exponent    ::=  ("e" | "E") ["+" | "-"] digitpart
floatnumber ::=  number [exponent]
floatvalue  ::=  [sign] (floatnumber | infinity | nan)

digitpart should not include an exclamation point or backtick quotes. The backticks appear to be the underlying syntax failing to render, but the exclamation points have a meaning, which is an issue. This is the underlying syntax:

sign: "+" | "-"
infinity: "Infinity" | "inf"
nan: "nan"
digitpart: `!digit` (["_"] `!digit`)*
number: [`digitpart`] "." `digitpart` | `digitpart` ["."]
exponent: ("e" | "E") ["+" | "-"] `digitpart`
floatnumber: number [`exponent`]

EBNF includes a "Special Sequences" syntax, that would be appropriate for this situation, but I'm uncertain if it is possible to use in the underlying YAML format. It should be rendered like this:

sign        ::=  "+" | "-"
infinity    ::=  "Infinity" | "inf"
nan         ::=  "nan"
digit       ::=  ? Any character in the unicode digit class ?
digitpart   ::=  digit (["_"] digit)* 
number      ::=  [digitpart] "." digitpart | digitpart ["."]
exponent    ::=  ("e" | "E") ["+" | "-"] digitpart
floatnumber ::=  number [exponent]
floatvalue  ::=  [sign] (floatnumber | infinity | nan)

Linked PRs

• gh-114070: correct the specification of digit in the float() docs #114080
• [3.12] gh-114070: correct the specification of digit in the float() docs (GH-114080) #114094
• [3.11] gh-114070: correct the specification of digit in the float() docs (GH-114080) #114095
• gh-114070: fix token reference warnings in expressions.rst #114169
• [3.12] gh-114070: fix token reference warnings in expressions.rst (GH-114169) #114192
• [3.11] gh-114070: fix token reference warnings in expressions.rst (GH-114169) #114193

[skirpichev]

It should be rendered like this

In other places - angle brackets are used to surround RHS's of digit-like definitions. For example:

• https://docs.python.org/3/reference/lexical_analysis.html#identifiers
• https://docs.python.org/3/library/string.html#format-string-syntax

[hugovk]

• docs.python.org/3/library/string.html#format-string-syntax

This one has a clickable digit link, which takes us to:

digit        ::=  "0"..."9"

In YAML:

   digit: "0"..."9"

Would it be appropriate to add something like that here?

[skirpichev]

@hugovk, no - that will be incorrect in this case. This is valid, for example, in case of floating-point literals.

But the float() constructor (int() too) supports more (not sure for good or evil): "Here digit is a Unicode decimal digit (character in the Unicode general category Nd)." (c)

For example:

>>> float('١٢٣')  # arabic decimals: U+0661-3
123.0
>>> int('١٢٣')
123

[hugovk]

OK, then shall we use angle brackets here?

[skirpichev]

I think so. The documentation says it's using "a modified Backus–Naur form (BNF) grammar notation", where: "A phrase between angular brackets (<...>) gives an informal description of the symbol defined; e.g., this could be used to describe the notion of ‘control character’ if needed."

[hauntsaninja]

I think the markup got regressed in https://github.com/python/cpython/pull/112669/files#diff-6a7a07ac473fdd76734669b1b70626ad2176011129902f6add017810f54d0439L667

I improved the grammar here in #100437 , let me know if there's anything I can help with. Moving the description of digit into angle brackets within the grammar rule sounds reasonable to me.

[skirpichev]

@hauntsaninja, see #114080