Possible unnecessary OverflowError in random.getrandbits

[Nokhrin]

Documentation

Problem

Actual upper bound of a random.Random.randbytes is 2**28 - 1

Following docs and exception message I would expect the limit of 2 ** 32 - 1 as for C 4-bytes integer

    ## -------------------- bytes methods ---------------------

    def randbytes(self, n):
        """Generate n random bytes."""
        return self.getrandbits(n * 8).to_bytes(n, 'little')

>>> bytes_string = random.randbytes(2 ** 28)
Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\code.py", line 90, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
  File "C:\Program Files\Python39\lib\random.py", line 286, in randbytes
    return self.getrandbits(n * 8).to_bytes(n, 'little')
OverflowError: Python int too large to convert to C int
>>> bytes_string = random.randbytes(2 ** 28 - 1)
>>> len(bytes_string)
268435455

Reproduced on
Python 3.9.4 (tags/v3.9.4:1f2e308, Apr 6 2021, 13:40:21) [MSC v.1928 64 bit (AMD64)] on win32
Python 3.11.2 (main, Jul 19 2024, 12:24:02) [GCC 12.2.0] on linux

Will you please point me - if there is a bug in docs/implementation, or this is my misunderstanding only.
Thank you in advance for you patience.

Linked PRs

• gh-133489: Document random.randbytes bounds. #133529
• gh-133489: Remove size restrictions on getrandbits() and randbytes() #133658
• [3.14] gh-133489: Remove size restrictions on getrandbits() and randbytes() (GH-133658) #134964
• [3.13] gh-133489: Remove size restrictions on getrandbits() and randbytes() (GH-133658) #134965

[rhettinger]

@serhiy-storchaka The overflow occurs in the upstream call to random.getrandbits(8 * 2 ** 28 - 1).

The argument clinic spec reads:

_random.Random.getrandbits

  self: self(type="RandomObject *")
  k: int
  /

Should k be switched to object and converted with PyLong_AsLongLongAndOverflow? The 32-bit signed int limitation on Windows seems unnecessarily restrictive.

[skirpichev]

Why not have int32 converter in the AC? Now we have PyLong_AsInt32/AsUInt32/etc.

[rhettinger]

@skirpichev We want something like ssize_t so that we can ask for arrays as big as we can allocate. The limit should be driven by memory constraints rather than function argument constraints. 32-bit signed is insufficient.