Doctest fails in `stdtypes.rst`

[CharlieZhao95]

Documentation

When some codes are modified, the doc and doctests are not updated synchronously.

Local doctest:

./python -m doctest ./Doc/library/stdtypes.rst

Details

**********************************************************************
File "./Doc/library/stdtypes.rst", line 3769, in stdtypes.rst
Failed example:
    v[1:4]
Expected:
    <memory at 0x7f3ddc9f4350>
Got:
    <memory at 0x7f42c250f040>
**********************************************************************
File "./Doc/library/stdtypes.rst", line 3952, in stdtypes.rst
Failed example:
    mm.tolist()
Expected:
    [89, 98, 99]
Got:
    [97, 98, 99]
**********************************************************************
File "./Doc/library/stdtypes.rst", line 4037, in stdtypes.rst
Failed example:
    x[0] = b'a'
Expected:
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: memoryview: invalid value for format "B"
Got:
    Traceback (most recent call last):
      File "/home/github/cpython/Lib/doctest.py", line 1357, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest stdtypes.rst[234]>", line 1, in <module>
        x[0] = b'a'
        ~^^^
    TypeError: memoryview: invalid type for format 'B'
**********************************************************************
File "./Doc/library/stdtypes.rst", line 4789, in stdtypes.rst
Failed example:
    keys ^ {'sausage', 'juice'}
Expected:
    {'juice', 'sausage', 'bacon', 'spam'}
Got:
    {'spam', 'sausage', 'bacon', 'juice'}
**********************************************************************
File "./Doc/library/stdtypes.rst", line 4791, in stdtypes.rst
Failed example:
    keys | ['juice', 'juice', 'juice']
Expected:
    {'juice', 'sausage', 'bacon', 'spam', 'eggs'}
Got:
    {'spam', 'bacon', 'juice'}
**********************************************************************
File "./Doc/library/stdtypes.rst", line 4997, in stdtypes.rst
Failed example:
    dict[str][str]
Expected:
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: There are no type variables left in dict[str]
Got:
    Traceback (most recent call last):
      File "/home/github/cpython/Lib/doctest.py", line 1357, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest stdtypes.rst[335]>", line 1, in <module>
        dict[str][str]
        ~~~~~~~~~^^^^^
    TypeError: dict[str] is not a generic class
**********************************************************************
File "./Doc/library/stdtypes.rst", line 5209, in stdtypes.rst
Failed example:
    isinstance(1, int | list[int])
Expected:
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: isinstance() argument 2 cannot contain a parameterized generic
Got:
    True
**********************************************************************
File "./Doc/library/stdtypes.rst", line 5514, in stdtypes.rst
Failed example:
    int.__subclasses__()
Expected:
    [<class 'bool'>]
Got:
    [<class 'bool'>, <enum 'IntEnum'>, <flag 'IntFlag'>, <class 're._constants._NamedIntConstant'>]
**********************************************************************
File "./Doc/library/stdtypes.rst", line 5548, in stdtypes.rst
Failed example:
    _ = int('2' * 5432)
Expected:
    Traceback (most recent call last):
    ...
    ValueError: Exceeds the limit (4300 digits) for integer string conversion: value has 5432 digits; use sys.set_int_max_str_digits() to increase the limit.
Got:
    Traceback (most recent call last):
      File "/home/github/cpython/Lib/doctest.py", line 1357, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest stdtypes.rst[361]>", line 1, in <module>
        _ = int('2' * 5432)
            ^^^^^^^^^^^^^^^
    ValueError: Exceeds the limit (4300 digits) for integer string conversion: value has 5432 digits; use sys.set_int_max_str_digits() to increase the limit
**********************************************************************
File "./Doc/library/stdtypes.rst", line 5556, in stdtypes.rst
Failed example:
    len(str(i_squared))
Expected:
    Traceback (most recent call last):
    ...
    ValueError: Exceeds the limit (4300 digits) for integer string conversion: value has 8599 digits; use sys.set_int_max_str_digits() to increase the limit.
Got:
    Traceback (most recent call last):
      File "/home/github/cpython/Lib/doctest.py", line 1357, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest stdtypes.rst[365]>", line 1, in <module>
        len(str(i_squared))
            ^^^^^^^^^^^^^^
    ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit
**********************************************************************

1 items had failures:
  10 of 374 in stdtypes.rst
***Test Failed*** 10 failures.

BTW, It seems that all doctests are passing in the CI pipeline. Those failed cases are not found by CI doctest.
See: https://github.com/CharlieZhao95/cpython/actions/runs/5461321900/jobs/9939195566

...
Document: library/stdtypes
--------------------------
1 items passed all tests:
  50 tests in default
50 tests in 1 items.
50 passed and 0 failed.
Test passed.

Environment:

CPython versions tested on: 3.13.0a0
Operating system and architecture: Ubuntu 22.04.1 LTS

Linked PRs

• gh-106446: Fix failed doctest in stdtypes #106447
• [3.12] gh-106446: Fix failed doctest in stdtypes (GH-106447) #106741
• [3.11] gh-106446: Fix failed doctest in stdtypes (GH-106447) #106742

[CharlieZhao95]

Failed doctests in stdtypes.rst:

• Memory Views (3 failed)
• Dictionary view objects (2 failed)
• Generic Alias Type (1 failed)
• Union Type (1 failed)
• Special Attributes (1 failed)
• Integer string conversion length limitation (2 failed)

[terryjreedy]

I get 15 failures on Windows; the additional 5 (3,4,5,6,8 in details) seem to be from long ints being 8 versus 4 bytes.

Details, with error numbers added

f:\dev\3x>python -m doctest Doc/library/stdtypes.rst
Running Debug|x64 interpreter...
1 **********************************************************************
File "Doc/library/stdtypes.rst", line 3769, in stdtypes.rst
Failed example:
    v[1:4]
Expected:
    <memory at 0x7f3ddc9f4350>
Got:
    <memory at 0x00000183A55BEA90>
2 **********************************************************************
File "Doc/library/stdtypes.rst", line 3952, in stdtypes.rst
Failed example:
    mm.tolist()
Expected:
    [89, 98, 99]
Got:
    [97, 98, 99]
3 **********************************************************************
File "Doc/library/stdtypes.rst", line 4017, in stdtypes.rst
Failed example:
    x.itemsize
Expected:
    8
Got:
    4
4 **********************************************************************
File "Doc/library/stdtypes.rst", line 4021, in stdtypes.rst
Failed example:
    x.nbytes
Expected:
    24
Got:
    12
5 **********************************************************************
File "Doc/library/stdtypes.rst", line 4028, in stdtypes.rst
Failed example:
    len(y)
Expected:
    24
Got:
    12
6 **********************************************************************
File "Doc/library/stdtypes.rst", line 4030, in stdtypes.rst
Failed example:
    y.nbytes
Expected:
    24
Got:
    12
7 **********************************************************************
File "Doc/library/stdtypes.rst", line 4037, in stdtypes.rst
Failed example:
    x[0] = b'a'
Expected:
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: memoryview: invalid value for format "B"
Got:
    Traceback (most recent call last):
      File "f:\dev\3x\Lib\doctest.py", line 1357, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest stdtypes.rst[234]>", line 1, in <module>
        x[0] = b'a'
        ~^^^
    TypeError: memoryview: invalid type for format 'B'
8 **********************************************************************
File "Doc/library/stdtypes.rst", line 4079, in stdtypes.rst
Failed example:
    y.nbytes
Expected:
    48
Got:
    24
9 **********************************************************************
File "Doc/library/stdtypes.rst", line 4789, in stdtypes.rst
Failed example:
    keys ^ {'sausage', 'juice'}
Expected:
    {'juice', 'sausage', 'bacon', 'spam'}
Got:
    {'sausage', 'juice', 'bacon', 'spam'}
10 **********************************************************************
File "Doc/library/stdtypes.rst", line 4791, in stdtypes.rst
Failed example:
    keys | ['juice', 'juice', 'juice']
Expected:
    {'juice', 'sausage', 'bacon', 'spam', 'eggs'}
Got:
    {'juice', 'bacon', 'spam'}
11 **********************************************************************
File "Doc/library/stdtypes.rst", line 4997, in stdtypes.rst
Failed example:
    dict[str][str]
Expected:
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: There are no type variables left in dict[str]
Got:
    Traceback (most recent call last):
      File "f:\dev\3x\Lib\doctest.py", line 1357, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest stdtypes.rst[335]>", line 1, in <module>
        dict[str][str]
        ~~~~~~~~~^^^^^
    TypeError: dict[str] is not a generic class
12 **********************************************************************
File "Doc/library/stdtypes.rst", line 5209, in stdtypes.rst
Failed example:
    isinstance(1, int | list[int])
Expected:
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: isinstance() argument 2 cannot contain a parameterized generic
Got:
    True
13 **********************************************************************
File "Doc/library/stdtypes.rst", line 5514, in stdtypes.rst
Failed example:
    int.__subclasses__()
Expected:
    [<class 'bool'>]
Got:
    [<class 'bool'>, <enum 'IntEnum'>, <flag 'IntFlag'>, <class 're._constants._NamedIntConstant'>]
14 **********************************************************************
File "Doc/library/stdtypes.rst", line 5548, in stdtypes.rst
Failed example:
    _ = int('2' * 5432)
Expected:
    Traceback (most recent call last):
    ...
    ValueError: Exceeds the limit (4300 digits) for integer string conversion: value has 5432 digits; use sys.set_int_max_str_digits() to increase the limit.
Got:
    Traceback (most recent call last):
      File "f:\dev\3x\Lib\doctest.py", line 1357, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest stdtypes.rst[361]>", line 1, in <module>
        _ = int('2' * 5432)
            ^^^^^^^^^^^^^^^
    ValueError: Exceeds the limit (4300 digits) for integer string conversion: value has 5432 digits; use sys.set_int_max_str_digits() to increase the limit
15 **********************************************************************
File "Doc/library/stdtypes.rst", line 5556, in stdtypes.rst
Failed example:
    len(str(i_squared))
Expected:
    Traceback (most recent call last):
    ...
    ValueError: Exceeds the limit (4300 digits) for integer string conversion: value has 8599 digits; use sys.set_int_max_str_digits() to increase the limit.
Got:
    Traceback (most recent call last):
      File "f:\dev\3x\Lib\doctest.py", line 1357, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest stdtypes.rst[365]>", line 1, in <module>
        len(str(i_squared))
            ^^^^^^^^^^^^^^
    ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit
**********************************************************************
1 items had failures:
  15 of 374 in stdtypes.rst
***Test Failed*** 15 failures.

I don't know how CI is run to avoid failures; perhaps it skips some things by examining failure. But some 'expected's are incorrect and should be changed in any case.

Comments: (I assume that you got same non-matching expected on Ubuntu. I checked validity of replacement expecteds back to 3.11 to be sure we can backport.)

1. Address could be 0x..., but this would require option and comment. Maybe CI ignores 0x... differences; leave alone.
2. list(b'abc'): 97, not 89 is correct for ord('a'). Fix.
3. Rather sure int size issue, Linux vs Windows. Leave as CI run on Ubuntu.
4. ""
5. ""
6. ""
7. The exception has been TypeError since at least 3.11. Fix.
8. see 3.
9. Set comparison fails only on order difference. Standard workaround is to convert set to list, as done in 2.
   Perhaps CI does so. Leave alone as long as CI passes as answer is correct from reader viewpoint.
10. This is bizarre. First, set operators require 'other' to also be a set. Perhaps a set was used originally, then changed to a list to show that or-ing with 'juice' multiple times has the same effect as doing so once, without actually rerunning the code. Second, keys originally had 4 items, 2 were deleted, leaving 2. Perhaps keys | {'juice'} was originally before the deletions at which time the answer was correct. Change the test to >>> keys | {'juice'} and fix the answer.
11. Message has been 'not a generic class' since at least 3.11. Fix.
12. True is correct since at least 3.11. Fix.
13. int has 4 subclasses since at least. Fix.
14. 'Got' same in 3.11. Already in PR.
15. ""

[CharlieZhao95]

I get the same result on Windows. Please let me know if further testing or other work is required, so as not to duplicate the work you are doing~

[terryjreedy]

I finished testing, but will 'let' you revise the PR.

[CharlieZhao95]

Some additions:

(1, 9) The doctest docs recommend a solution to check set and address. Maybe we could use this to improve docs.

10. Perhaps keys | {'juice'} was originally before the deletions at which time the answer was correct. Change the test to >>> keys | {'juice'} and fix the answer.

(10) The keys here is not a real set object, but a set-like object (dict_keys). The test cases for operators between set-like object and list are intentionally added. See gh-96408 for details. So we just need to fix answer :)

12. True is correct since at least 3.11. Fix.

(12) This result is correct until Python 3.10. But please consider the following examples:

# Example1: In python 3.10
>>> isinstance(1, int | list[int])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: isinstance() argument 2 cannot contain a parameterized generic

    
# Example2: In python 3.13.0a0
>>> isinstance(1, int | list[int])  # short-circuit evaluation
True

# Example3: In python 3.13.0a0
>>> isinstance([1], int | list[int])
Traceback (most recent call last):
  ...
TypeError: isinstance() argument 2 cannot be a parameterized generic

I guess the syntax of Python has changed. In example 2, the second operand is never evaluated (short-circuit). Only modifying the result to True does not match the description in docs. It might be better to add example 3 and update docs.