gh-141984: Reword and reorganize the first part of Atoms docs (GH-144117)

Co-authored-by: Blaise Pabon <blaise@gmail.com>
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>

Author: 3 people

Doc/faq/programming.rst

@@ -1852,6 +1852,8 @@ to the object:
 13891296
 
 
+.. _faq-identity-with-is:
+
 When can I rely on identity tests with the *is* operator?
 ---------------------------------------------------------
 

Doc/library/stdtypes.rst

@@ -265,9 +265,17 @@ The constructors :func:`int`, :func:`float`, and
    pair: operator; % (percent)
    pair: operator; **
 
+.. _stdtypes-mixed-arithmetic:
+
 Python fully supports mixed arithmetic: when a binary arithmetic operator has
-operands of different numeric types, the operand with the "narrower" type is
-widened to that of the other, where integer is narrower than floating point.
+operands of different built-in numeric types, the operand with the "narrower"
+type is widened to that of the other:
+
+* If both arguments are complex numbers, no conversion is performed;
+* if either argument is a complex or a floating-point number, the other is
+  converted to a floating-point number;
+* otherwise, both must be integers and no conversion is necessary.
+
 Arithmetic with complex and real operands is defined by the usual mathematical
 formula, for example::
 

Doc/library/token.rst

@@ -50,8 +50,7 @@ The token constants are:
 
 .. data:: NAME
 
-   Token value that indicates an :ref:`identifier <identifiers>`.
-   Note that keywords are also initially tokenized as ``NAME`` tokens.
+   Token value that indicates an :ref:`identifier or keyword <identifiers>`.
 
 .. data:: NUMBER
 

Doc/reference/expressions.rst

@@ -9,9 +9,11 @@ Expressions
 
 This chapter explains the meaning of the elements of expressions in Python.
 
-**Syntax Notes:** In this and the following chapters, extended BNF notation will
-be used to describe syntax, not lexical analysis.  When (one alternative of) a
-syntax rule has the form
+**Syntax Notes:** In this and the following chapters,
+:ref:`grammar notation <notation>` will be used to describe syntax,
+not lexical analysis.
+
+When (one alternative of) a syntax rule has the form:
 
 .. productionlist:: python-grammar
    name: othername
@@ -29,17 +31,13 @@ Arithmetic conversions
 
 When a description of an arithmetic operator below uses the phrase "the numeric
 arguments are converted to a common real type", this means that the operator
-implementation for built-in types works as follows:
-
-* If both arguments are complex numbers, no conversion is performed;
-
-* if either argument is a complex or a floating-point number, the other is converted to a floating-point number;
-
-* otherwise, both must be integers and no conversion is necessary.
+implementation for built-in numeric types works as described in the
+:ref:`Numeric Types <stdtypes-mixed-arithmetic>` section of the standard
+library documentation.
 
-Some additional rules apply for certain operators (e.g., a string as a left
-argument to the '%' operator).  Extensions must define their own conversion
-behavior.
+Some additional rules apply for certain operators and non-numeric operands
+(for example, a string as a left argument to the ``%`` operator).
+Extensions must define their own conversion behavior.
 
 
 .. _atoms:
@@ -49,15 +47,57 @@ Atoms
 
 .. index:: atom
 
-Atoms are the most basic elements of expressions.  The simplest atoms are
-identifiers or literals.  Forms enclosed in parentheses, brackets or braces are
-also categorized syntactically as atoms.  The syntax for atoms is:
+Atoms are the most basic elements of expressions.
+The simplest atoms are :ref:`names <identifiers>` or literals.
+Forms enclosed in parentheses, brackets or braces are also categorized
+syntactically as atoms.
 
-.. productionlist:: python-grammar
-   atom: `identifier` | `literal` | `enclosure`
-   enclosure: `parenth_form` | `list_display` | `dict_display` | `set_display`
-            : | `generator_expression` | `yield_atom`
+Formally, the syntax for atoms is:
+
+.. grammar-snippet::
+   :group: python-grammar
+
+   atom:
+      | 'True'
+      | 'False'
+      | 'None'
+      | '...'
+      | `identifier`
+      | `literal`
+      | `enclosure`
+   enclosure:
+      | `parenth_form`
+      | `list_display`
+      | `dict_display`
+      | `set_display`
+      | `generator_expression`
+      | `yield_atom`
+
+
+.. _atom-singletons:
+
+Built-in constants
+------------------
+
+The keywords ``True``, ``False``, and ``None`` name
+:ref:`built-in constants <built-in-consts>`.
+The token ``...`` names the :py:data:`Ellipsis` constant.
 
+Evaluation of these atoms yields the corresponding value.
+
+.. note::
+
+   Several more built-in constants are available as global variables,
+   but only the ones mentioned here are :ref:`keywords <keywords>`.
+   In particular, these names cannot be reassigned or used as attributes:
+
+   .. code-block:: pycon
+
+      >>> False = 123
+        File "<input>", line 1
+         False = 123
+         ^^^^^
+      SyntaxError: cannot assign to False
 
 .. _atom-identifiers:
 
@@ -131,51 +171,104 @@ Literals
 
 .. index:: single: literal
 
-Python supports string and bytes literals and various numeric literals:
+A :dfn:`literal` is a textual representation of a value.
+Python supports numeric, string and bytes literals.
+:ref:`Format strings <f-strings>` and :ref:`template strings <t-strings>`
+are treated as string literals.
+
+Numeric literals consist of a single :token:`NUMBER <python-grammar:NUMBER>`
+token, which names an integer, floating-point number, or an imaginary number.
+See the :ref:`numbers` section in Lexical analysis documentation for details.
+
+String and bytes literals may consist of several tokens.
+See section :ref:`string-concatenation` for details.
+
+Note that negative and complex numbers, like ``-3`` or ``3+4.2j``,
+are syntactically not literals, but :ref:`unary <unary>` or
+:ref:`binary <binary>` arithmetic operations involving the ``-`` or ``+``
+operator.
+
+Evaluation of a literal yields an object of the given type
+(:class:`int`, :class:`float`, :class:`complex`, :class:`str`,
+:class:`bytes`, or :class:`~string.templatelib.Template`) with the given value.
+The value may be approximated in the case of floating-point
+and imaginary literals.
+
+The formal grammar for literals is:
 
 .. grammar-snippet::
    :group: python-grammar
 
    literal: `strings` | `NUMBER`
 
-Evaluation of a literal yields an object of the given type (string, bytes,
-integer, floating-point number, complex number) with the given value.  The value
-may be approximated in the case of floating-point and imaginary (complex)
-literals.
-See section :ref:`literals` for details.
-See section :ref:`string-concatenation` for details on ``strings``.
-
 
 .. index::
    triple: immutable; data; type
    pair: immutable; object
 
+Literals and object identity
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
 All literals correspond to immutable data types, and hence the object's identity
 is less important than its value.  Multiple evaluations of literals with the
 same value (either the same occurrence in the program text or a different
 occurrence) may obtain the same object or a different object with the same
 value.
 
+.. admonition:: CPython implementation detail
+
+   For example, in CPython, *small* integers with the same value evaluate
+   to the same object::
+
+      >>> x = 7
+      >>> y = 7
+      >>> x is y
+      True
+
+   However, large integers evaluate to different objects::
+
+      >>> x = 123456789
+      >>> y = 123456789
+      >>> x is y
+      False
+
+   This behavior may change in future versions of CPython.
+   In particular, the boundary between "small" and "large" integers has
+   already changed in the past.
+
+   CPython will emit a :py:exc:`SyntaxWarning` when you compare literals
+   using ``is``::
+
+      >>> x = 7
+      >>> x is 7
+      <input>:1: SyntaxWarning: "is" with 'int' literal. Did you mean "=="?
+      True
+
+   See :ref:`faq-identity-with-is` for more information.
+
+:ref:`Template strings <t-strings>` are immutable but may reference mutable
+objects as :class:`~string.templatelib.Interpolation` values.
+For the purposes of this section, two t-strings have the "same value" if
+both their structure and the *identity* of the values match.
+
+.. impl-detail::
+
+   Currently, each evaluation of a template string results in
+   a different object.
+
 
 .. _string-concatenation:
 
 String literal concatenation
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Multiple adjacent string or bytes literals (delimited by whitespace), possibly
+Multiple adjacent string or bytes literals, possibly
 using different quoting conventions, are allowed, and their meaning is the same
 as their concatenation::
 
    >>> "hello" 'world'
    "helloworld"
 
-Formally:
-
-.. grammar-snippet::
-   :group: python-grammar
-
-   strings: ( `STRING` | `fstring`)+ | `tstring`+
-
 This feature is defined at the syntactical level, so it only works with literals.
 To concatenate string expressions at run time, the '+' operator may be used::
 
@@ -208,6 +301,13 @@ string literals::
    >>> t"Hello" t"{name}!"
    Template(strings=('Hello', '!'), interpolations=(...))
 
+Formally:
+
+.. grammar-snippet::
+   :group: python-grammar
+
+   strings: (`STRING` | `fstring`)+ | `tstring`+
+
 
 .. _parenthesized:
 
@@ -1390,8 +1490,9 @@ for the operands): ``-1**2`` results in ``-1``.
 
 The power operator has the same semantics as the built-in :func:`pow` function,
 when called with two arguments: it yields its left argument raised to the power
-of its right argument.  The numeric arguments are first converted to a common
-type, and the result is of that type.
+of its right argument.
+Numeric arguments are first :ref:`converted to a common type <stdtypes-mixed-arithmetic>`,
+and the result is of that type.
 
 For int operands, the result has the same type as the operands unless the second
 argument is negative; in that case, all arguments are converted to float and a
@@ -1477,9 +1578,10 @@ operators and one for additive operators:
 
 The ``*`` (multiplication) operator yields the product of its arguments.  The
 arguments must either both be numbers, or one argument must be an integer and
-the other must be a sequence. In the former case, the numbers are converted to a
-common real type and then multiplied together.  In the latter case, sequence
-repetition is performed; a negative repetition factor yields an empty sequence.
+the other must be a sequence. In the former case, the numbers are
+:ref:`converted to a common real type <stdtypes-mixed-arithmetic>` and then
+multiplied together.  In the latter case, sequence repetition is performed;
+a negative repetition factor yields an empty sequence.
 
 This operation can be customized using the special :meth:`~object.__mul__` and
 :meth:`~object.__rmul__` methods.
@@ -1507,7 +1609,8 @@ This operation can be customized using the special :meth:`~object.__matmul__` an
    pair: operator; //
 
 The ``/`` (division) and ``//`` (floor division) operators yield the quotient of
-their arguments.  The numeric arguments are first converted to a common type.
+their arguments.  The numeric arguments are first
+:ref:`converted to a common type <stdtypes-mixed-arithmetic>`.
 Division of integers yields a float, while floor division of integers results in an
 integer; the result is that of mathematical division with the 'floor' function
 applied to the result.  Division by zero raises the :exc:`ZeroDivisionError`
@@ -1523,8 +1626,9 @@ The floor division operation can be customized using the special
    pair: operator; % (percent)
 
 The ``%`` (modulo) operator yields the remainder from the division of the first
-argument by the second.  The numeric arguments are first converted to a common
-type.  A zero right argument raises the :exc:`ZeroDivisionError` exception.  The
+argument by the second.  The numeric arguments are first
+:ref:`converted to a common type <stdtypes-mixed-arithmetic>`.
+A zero right argument raises the :exc:`ZeroDivisionError` exception.  The
 arguments may be floating-point numbers, e.g., ``3.14%0.7`` equals ``0.34``
 (since ``3.14`` equals ``4*0.7 + 0.34``.)  The modulo operator always yields a
 result with the same sign as its second operand (or zero); the absolute value of
@@ -1555,7 +1659,9 @@ floating-point number using the :func:`abs` function if appropriate.
 
 The ``+`` (addition) operator yields the sum of its arguments.  The arguments
 must either both be numbers or both be sequences of the same type.  In the
-former case, the numbers are converted to a common real type and then added together.
+former case, the numbers are
+:ref:`converted to a common real type <stdtypes-mixed-arithmetic>` and then
+added together.
 In the latter case, the sequences are concatenated.
 
 This operation can be customized using the special :meth:`~object.__add__` and
@@ -1570,8 +1676,9 @@ This operation can be customized using the special :meth:`~object.__add__` and
    single: operator; - (minus)
    single: - (minus); binary operator
 
-The ``-`` (subtraction) operator yields the difference of its arguments.  The
-numeric arguments are first converted to a common real type.
+The ``-`` (subtraction) operator yields the difference of its arguments.
+The numeric arguments are first
+:ref:`converted to a common real type <stdtypes-mixed-arithmetic>`.
 
 This operation can be customized using the special :meth:`~object.__sub__` and
 :meth:`~object.__rsub__` methods.