3.13.0a4 SIGSEGV when calling next on a list-iterator in for that loops the same iterator

[hroncok]

Bug report

Bug description:

---

EDIT smaller reproducer:

parts = iter([...] * 3)  # odd number of elements

for part in parts:
    next(parts, ...)

---

Hello, when we try to build Flask in Fedora with Python 3.13.0a4, we see a SIGSEGV. With @befeleme we've been able to isolate the failure to the following:

# reporducer.py

import flask
app = flask.Flask("flask_test")
app.config.update(TESTING=True)

@app.route("/")
def index():
    return "x"

with app.app_context():
    app.test_client().get("/")

$ sudo dnf --enablerepo=updates-testing install python3.13-debug
$ sudo dnf --enablerepo=updates-testing debuginfo-install python3.13

$ python3.13d --version
Python 3.13.0a4

$ python3.13d -m venv venv3.13d
$ . venv3.13d/bin/activate
(venv3.13d)$ pip install flask
...
(venv3.13d)$ pip list
Package      Version
------------ -------
blinker      1.7.0
click        8.1.7
Flask        3.0.2
itsdangerous 2.1.2
Jinja2       3.1.3
MarkupSafe   2.1.5
pip          23.2.1
Werkzeug     3.0.1

(venv3.13d)$ rm venv3.13d/lib64/python3.13/site-packages/markupsafe/_speedups.cpython-313-x86_64-linux-gnu.so  # to eliminate possibility of SIGSEGV in markupsafe

(venv3.13d)$ python -X dev reproducer.py
Fatal Python error: Segmentation fault

Current thread 0x00007f9445ba8740 (most recent call first):
  File "/tmp/venv3.13d/lib64/python3.13/site-packages/werkzeug/urls.py", line 40 in _unquote_partial
  File "/tmp/venv3.13d/lib64/python3.13/site-packages/werkzeug/urls.py", line 85 in uri_to_iri
  File "/tmp/venv3.13d/lib64/python3.13/site-packages/werkzeug/sansio/utils.py", line 137 in get_current_url
  File "/tmp/venv3.13d/lib64/python3.13/site-packages/werkzeug/wsgi.py", line 66 in get_current_url
  File "/tmp/venv3.13d/lib64/python3.13/site-packages/werkzeug/test.py", line 944 in _add_cookies_to_wsgi
  File "/tmp/venv3.13d/lib64/python3.13/site-packages/werkzeug/test.py", line 985 in run_wsgi_app
  File "/tmp/venv3.13d/lib64/python3.13/site-packages/werkzeug/test.py", line 1114 in open
  File "/tmp/venv3.13d/lib64/python3.13/site-packages/flask/testing.py", line 235 in open
  File "/tmp/venv3.13d/lib64/python3.13/site-packages/werkzeug/test.py", line 1160 in get
  File "/tmp/reproducer.py", line 10 in <module>
Segmentation fault (core dumped)

Here's a bt from gdb:

(gdb) bt
#0  0x00007ffff7af344f in Py_TYPE (ob=0x0) at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Include/object.h:333
#1  0x00007ffff7af3f7c in PyList_GET_SIZE (op=0x0) at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Include/cpython/listobject.h:31
#2  0x00007ffff7b04e26 in _PyEval_EvalFrameDefault (tstate=0x7ffff7f2a8b8 <_PyRuntime+247704>, frame=0x7ffff7fb8620, throwflag=0)
    at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Python/generated_cases.c.h:2544
#3  0x00007ffff7af53d3 in _PyEval_EvalFrame (tstate=0x7ffff7f2a8b8 <_PyRuntime+247704>, frame=0x7ffff7fb8120, throwflag=0)
    at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Include/internal/pycore_ceval.h:115
#4  0x00007ffff7b25677 in _PyEval_Vector (tstate=0x7ffff7f2a8b8 <_PyRuntime+247704>, func=0x7fffe8570b90, locals=0x0, args=0x7fffe88d8fd0, argcount=2, 
    kwnames=('method',)) at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Python/ceval.c:1788
#5  0x00007ffff796cfe2 in _PyFunction_Vectorcall (func=<function at remote 0x7fffe8570b90>, stack=0x7fffe88d8fd0, nargsf=2, kwnames=('method',))
    at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Objects/call.c:413
#6  0x00007ffff7970daa in _PyObject_VectorcallTstate (tstate=0x7ffff7f2a8b8 <_PyRuntime+247704>, callable=<function at remote 0x7fffe8570b90>, 
    args=0x7fffe88d8fd0, nargsf=2, kwnames=('method',)) at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Include/internal/pycore_call.h:168
#7  0x00007ffff79714d6 in method_vectorcall (method=<method at remote 0x7fffe8dc0c50>, args=0x7fffe88d8fd8, nargsf=9223372036854775809, 
    kwnames=('method',)) at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Objects/classobject.c:62
#8  0x00007ffff796ca02 in _PyVectorcall_Call (tstate=0x7ffff7f2a8b8 <_PyRuntime+247704>, func=0x7ffff7971329 <method_vectorcall>, 
    callable=<method at remote 0x7fffe8dc0c50>, tuple=('/',), kwargs={'method': 'GET'})
    at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Objects/call.c:285
#9  0x00007ffff796cd00 in _PyObject_Call (tstate=0x7ffff7f2a8b8 <_PyRuntime+247704>, callable=<method at remote 0x7fffe8dc0c50>, args=('/',), 
    kwargs={'method': 'GET'}) at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Objects/call.c:348
#10 0x00007ffff796cddb in PyObject_Call (callable=<method at remote 0x7fffe8dc0c50>, args=('/',), kwargs={'method': 'GET'})
    at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Objects/call.c:373
#11 0x00007ffff7afdbbf in _PyEval_EvalFrameDefault (tstate=0x7ffff7f2a8b8 <_PyRuntime+247704>, frame=0x7ffff7fb8098, throwflag=0)
    at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Python/generated_cases.c.h:1250
#12 0x00007ffff7af53d3 in _PyEval_EvalFrame (tstate=0x7ffff7f2a8b8 <_PyRuntime+247704>, frame=0x7ffff7fb8020, throwflag=0)
    at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Include/internal/pycore_ceval.h:115
#13 0x00007ffff7b25677 in _PyEval_Vector (tstate=0x7ffff7f2a8b8 <_PyRuntime+247704>, func=0x7fffe9d4dcd0, 
    locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='/tmp/reproducer.py') at remote 0x7fffe9d60e20>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff74e3830>, '__file__': '/tmp/reproducer.py', '__cached__': None, 'flask': <module at remote 0x7fffe9dcc290>, 'app': <Flask(import_name='flask_test', _static_folder='static', _static_url_path=None, template_folder='templates', root_path='/tmp', cli=<AppGroup(name='flask_test', context_settings={}, callback=None, params=[], help=None, epilog=None, options_metavar='[OPTIONS]', short_help=None, add_help_option=True, no_args_is_help=True, hidden=False, deprecated=False, invoke_without_command=False, subcommand_metavar='COMMAND [ARGS]...', chain=False, _result_callback=None, commands={}) at remote 0x7fffe8709eb0>, view_functions={'static': <function at remote 0x7fffe8adb1d0>, 'index': <function at remote 0x7fffe87559d0>}, err...(truncated), args=0x0, 
    argcount=0, kwnames=0x0) at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Python/ceval.c:1788
#14 0x00007ffff7af6edd in PyEval_EvalCode (co=<code at remote 0x5555555ece50>, 
    globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='/tmp/reproducer.py') at remote 0x7fffe9d60e20>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff74e3830>, '__file__': '/tmp/reproducer.py', '__cached__': None, 'flask': <module at remote 0x7fffe9dcc290>, 'app': <Flask(import_name='flask_test', _static_folder='static', _static_url_path=None, template_folder='templates', root_path='/tmp', cli=<AppGroup(name='flask_test', context_settings={}, callback=None, params=[], help=None, epilog=None, options_metavar='[OPTIONS]', short_help=None, add_help_option=True, no_args_is_help=True, hidden=False, deprecated=False, invoke_without_command=False, subcommand_metavar='COMMAND [ARGS]...', chain=False, _result_callback=None, commands={}) at remote 0x7fffe8709eb0>, view_functions={'static': <function at remote 0x7fffe8adb1d0>, 'index': <function at remote 0x7fffe87559d0>}, err...(truncated), 
    locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='/tmp/reproducer.py') at remote 0x7fffe9d60e20>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff74e3830>, '__file__': '/tmp/reproducer.py', '__cached__': None, 'flask': <module at remote 0x7fffe9dcc290>, 'app': <Flask(import_name='flask_test', _static_folder='static', _static_url_path=None, template_folder='templates', root_path='/tmp', cli=<AppGroup(name='flask_test', context_settings={}, callback=None, params=[], help=None, epilog=None, options_metavar='[OPTIONS]', short_help=None, add_help_option=True, no_args_is_help=True, hidden=False, deprecated=False, invoke_without_command=False, subcommand_metavar='COMMAND [ARGS]...', chain=False, _result_callback=None, commands={}) at remote 0x7fffe8709eb0>, view_functions={'static': <function at remote 0x7fffe8adb1d0>, 'index': <function at remote 0x7fffe87559d0>}, err...(truncated))
    at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Python/ceval.c:592
#15 0x00007ffff7bcd505 in run_eval_code_obj (tstate=0x7ffff7f2a8b8 <_PyRuntime+247704>, co=0x5555555ece50, 
    globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='/tmp/reproducer.py') at remote 0x7fffe9d60e20>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff74e3830>, '__file__': '/tmp/reproducer.py', '__cached__': None, 'flask': <module at remote 0x7fffe9dcc290>, 'app': <Flask(import_name='flask_test', _static_folder='static', _static_url_path=None, template_folder='templates', root_path='/tmp', cli=<AppGroup(name='flask_test', context_settings={}, callback=None, params=[], help=None, epilog=None, options_metavar='[OPTIONS]', short_help=None, add_help_option=True, no_args_is_help=True, hidden=False, deprecated=False, invoke_without_command=False, subcommand_metavar='COMMAND [ARGS]...', chain=False, _result_callback=None, commands={}) at remote 0x7fffe8709eb0>, view_functions={'static': <function at remote 0x7fffe8adb1d0>, 'index': <function at remote 0x7fffe87559d0>}, err...(truncated), 
    locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='/tmp/reproducer.py') at remote 0x7fffe9d60e20>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff74e3830>, '__file__': '/tmp/reproducer.py', '__cached__': None, 'flask': <module at remote 0x7fffe9dcc290>, 'app': <Flask(import_name='flask_test', _static_folder='static', _static_url_path=None, template_folder='templates', root_path='/tmp', cli=<AppGroup(name='flask_test', context_settings={}, callback=None, params=[], help=None, epilog=None, options_metavar='[OPTIONS]', short_help=None, add_help_option=True, no_args_is_help=True, hidden=False, deprecated=False, invoke_without_command=False, subcommand_metavar='COMMAND [ARGS]...', chain=False, _result_callback=None, commands={}) at remote 0x7fffe8709eb0>, view_functions={'static': <function at remote 0x7fffe8adb1d0>, 'index': <function at remote 0x7fffe87559d0>}, err...(truncated))
    at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Python/pythonrun.c:1294
#16 0x00007ffff7bcd8d5 in run_mod (mod=0x55555561ffd0, filename='/tmp/reproducer.py', 
    globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='/tmp/reproducer.py') at remote 0x7fffe9d60e20>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff74e3830>, '__file__': '/tmp/reproducer.py', '__cached__': None, 'flask': <module at remote 0x7fffe9dcc290>, 'app': <Flask(import_name='flask_test', _static_folder='static', _static_url_path=None, template_folder='templates', root_path='/tmp', cli=<AppGroup(name='flask_test', context_settings={}, callback=None, params=[], help=None, epilog=None, options_metavar='[OPTIONS]', short_help=None, add_help_option=True, no_args_is_help=True, hidden=False, deprecated=False, invoke_without_command=False, subcommand_metavar='COMMAND [ARGS]...', chain=False, _result_callback=None, commands={}) at remote 0x7fffe8709eb0>, view_functions={'static': <function at remote 0x7fffe8adb1d0>, 'index': <function at remote 0x7fffe87559d0>}, err...(truncated), 
    locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='/tmp/reproducer.py') at remote 0x7fffe9d60e20>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff74e3830>, '__file__': '/tmp/reproducer.py', '__cached__': None, 'flask': <module at remote 0x7fffe9dcc290>, 'app': <Flask(import_name='flask_test', _static_folder='static', _static_url_path=None, template_folder='templates', root_path='/tmp', cli=<AppGroup(name='flask_test', context_settings={}, callback=None, params=[], help=None, epilog=None, options_metavar='[OPTIONS]', short_help=None, add_help_option=True, no_args_is_help=True, hidden=False, deprecated=False, invoke_without_command=False, subcommand_metavar='COMMAND [ARGS]...', chain=False, _result_callback=None, commands={}) at remote 0x7fffe8709eb0>, view_functions={'static': <function at remote 0x7fffe8adb1d0>, 'index': <function at remote 0x7fffe87559d0>}, err...(truncated), 
    flags=0x7fffffffd318, arena=0x7fffe9daf700, interactive_src=0x0, generate_new_source=0)
    at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Python/pythonrun.c:1379
#17 0x00007ffff7bcd2e2 in pyrun_file (fp=0x5555555851b0, filename='/tmp/reproducer.py', start=257, 
    globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='/tmp/reproducer.py') at remote 0x7fffe9d60e20>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff74e3830>, '__file__': '/tmp/reproducer.py', '__cached__': None, 'flask': <module at remote 0x7fffe9dcc290>, 'app': <Flask(import_name='flask_test', _static_folder='static', _static_url_path=None, template_folder='templates', root_path='/tmp', cli=<AppGroup(name='flask_test', context_settings={}, callback=None, params=[], help=None, epilog=None, options_metavar='[OPTIONS]', short_help=None, add_help_option=True, no_args_is_help=True, hidden=False, deprecated=False, invoke_without_command=False, subcommand_metavar='COMMAND [ARGS]...', chain=False, _result_callback=None, commands={}) at remote 0x7fffe8709eb0>, view_functions={'static': <function at remote 0x7fffe8adb1d0>, 'index': <function at remote 0x7fffe87559d0>}, err...(truncated), 
    locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='/tmp/reproducer.py') at remote 0x7fffe9d60e20>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff74e3830>, '__file__': '/tmp/reproducer.py', '__cached__': None, 'flask': <module at remote 0x7fffe9dcc290>, 'app': <Flask(import_name='flask_test', _static_folder='static', _static_url_path=None, template_folder='templates', root_path='/tmp', cli=<AppGroup(name='flask_test', context_settings={}, callback=None, params=[], help=None, epilog=None, options_metavar='[OPTIONS]', short_help=None, add_help_option=True, no_args_is_help=True, hidden=False, deprecated=False, invoke_without_command=False, subcommand_metavar='COMMAND [ARGS]...', chain=False, _result_callback=None, commands={}) at remote 0x7fffe8709eb0>, view_functions={'static': <function at remote 0x7fffe8adb1d0>, 'index': <function at remote 0x7fffe87559d0>}, err...(truncated), closeit=1, 
    flags=0x7fffffffd318) at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Python/pythonrun.c:1215
#18 0x00007ffff7bcb69d in _PyRun_SimpleFileObject (fp=0x5555555851b0, filename='/tmp/reproducer.py', closeit=1, flags=0x7fffffffd318)
    at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Python/pythonrun.c:464
#19 0x00007ffff7bcaa1c in _PyRun_AnyFileObject (fp=0x5555555851b0, filename='/tmp/reproducer.py', closeit=1, flags=0x7fffffffd318)
    at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Python/pythonrun.c:77
#20 0x00007ffff7c033cb in pymain_run_file_obj (program_name='/tmp/venv3.13d/bin/python', 
    filename='/tmp/reproducer.py', skip_source_first_line=0) at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Modules/main.c:357
#21 0x00007ffff7c034a5 in pymain_run_file (config=0x7ffff7f05838 <_PyRuntime+96024>)
    at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Modules/main.c:376
#22 0x00007ffff7c03d17 in pymain_run_python (exitcode=0x7fffffffd4b4) at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Modules/main.c:628
#23 0x00007ffff7c03e5c in Py_RunMain () at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Modules/main.c:707
#24 0x00007ffff7c03f32 in pymain_main (args=0x7fffffffd530) at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Modules/main.c:737
#25 0x00007ffff7c03ffa in Py_BytesMain (argc=4, argv=0x7fffffffd6a8) at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Modules/main.c:761
#26 0x000055555555517d in main (argc=4, argv=0x7fffffffd6a8) at /usr/src/debug/python3.13-3.13.0~a4-1.fc39.x86_64/Programs/python.c:15

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

• gh-115733: Fix crash involving exhausted list iterator #115740

[hroncok]

Flask-less reproducer with werkzeug only:

$ python -X dev -c 'import werkzeug; werkzeug.urls.uri_to_iri("/")'
Fatal Python error: Segmentation fault

Current thread 0x00007f61cf7bf740 (most recent call first):
  File "/tmp/venv3.13d/lib64/python3.13/site-packages/werkzeug/urls.py", line 40 in _unquote_partial
  File "/tmp/venv3.13d/lib64/python3.13/site-packages/werkzeug/urls.py", line 85 in uri_to_iri
  File "<string>", line 1 in <module>
Segmentation fault (core dumped)

[hroncok]

Werkzeug-less reproducer:

parts = iter([...] * 3)  # odd number of elements

for part in parts:
    next(parts, ...)

[hroncok]

When the iterator is made from a tuple, it does not crash. If it has an even number of elements, it does not crash (only small integers tested).

[hroncok]

This example (from Python 3.12) shows why odd-number of elements matter:

>>> parts = iter([1,2,3,4,5])
>>> 
>>> for part in parts:
...     print('P', part)
...     print('N', next(parts, ...))
... 
P 1
N 2
P 3
N 4
P 5
N Ellipsis
>>> parts = iter([1,2,3,4])
>>> 
>>> for part in parts:
...     print('P', part)
...     print('N', next(parts, ...))
... 
P 1
N 2
P 3
N 4

[hroncok]

Reproduces on the current main branch.

[hroncok]

Reverting a2d4281 makes it not crash.

cc @corona10