SystemError: compiler_lookup_arg(name='name_1') with reftype=7 failed in <genexpr>

[15r10nk]

Crash report

What happened?

I found the following issue by fuzzing with pysource-codegen.

This script failed to compile the given code to bytecode:

code="""
(name_3): name_5
name_4: (
    name_4
    async for (
        name_5
        for something in name_1
        async for () in (name_0 async for name_5 in name_0 for name_0 in name_1)
    ).name_3 in {name_5: name_5 for name_1 in name_4}
)
"""
compile(code,"<file>","exec")

running it with cpython results in the following error:

Traceback (most recent call last):
  File "/home/frank/projects/cpython/../pysource-playground/bug.py", line 12, in <module>
    compile(code,"<file>","exec")
    ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
SystemError: compiler_lookup_arg(name='name_1') with reftype=7 failed in <genexpr>; freevars of code <genexpr>: ('name_1',)

I bisected the problem down to 9b8611e @JelleZijlstra do you want to take a look at it?

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.14.0a0 (bisect/bad:9b8611eeea1, Apr 13 2025, 17:20:02) [GCC 12.2.0]

Linked PRs

• gh-132479: Fix crash with multiple comprehensions in annotations #132778

[JelleZijlstra]

Thanks! Slightly simplified repro (don't need the async):

code="""
x: name_5
name_4: (
    name_4
    for (
        name_5
        for something in name_1
        for () in (name_0 for name_5 in name_0 for name_0 in name_1)
    ).name_3 in {name_5 for name_1 in name_4}
)
"""
compile(code,"<file>","exec")

cc @carljm too since this involves a comprehension. It doesn't repro any more when I change the setcomp to a genexp (which isn't inlined).

[15r10nk]

I have found two other very similar errors. Perhaps this will help to solve the problem.

name_2: {}
name_0: {
    name_0: name_5
    for name_4 in [name_5 for name_0 in name_1]
    if ((name_0 for name_2 in name_5) for [] in name_1)
}

SystemError: compiler_lookup_arg(name='name_0') with reftype=3 failed in __annotate__; freevars of code <genexpr>: ('name_0',)

[][[]]: {name_4 for name_0 in name_1}
name_2: {
    something: (lambda: name_0 for [] in name_4 for name_2 in name_5)
    for something in ()
    if name_0
    for name_5 in something
}

SystemError: compiler_lookup_arg(name='name_0') with reftype=3 failed in __annotate__; freevars of code <genexpr>: ('name_0', 'name_5')

[carljm]

I would like to investigate this, but I'm afraid I won't realistically have time for it anytime soon, so I would encourage anyone interested to look into it.

[15r10nk]

@JelleZijlstra the new version of pysource-minimize can now also remove the await. I just released a new version you can see it here in action.

Please tell me if anyone finds another way how this can be simplified. Maybe it allows me to improve pysource-minimize a bit more.

[15r10nk]

I found some new minimization.

• expressions are replaced with 0 if possible
• names are replaced with an unique name if possible

This gave me the following new examples (minimized from the original (not minimized) 3 problems):

(unique_name_0): 0
unique_name_1: (
    0
    for (
        0
        for unique_name_2 in 0
        for () in (0 for unique_name_3 in unique_name_4 for unique_name_5 in name_1)
    ).name_3 in {0: 0 for name_1 in unique_name_8}
    if name_1
)

unique_name_0: 0
unique_name_1: {
    0: 0
    for unique_name_2 in [0 for name_0 in unique_name_4]
    if {
        0: 0
        for unique_name_5 in 0
        if name_0
        if ((name_0 for unique_name_8 in unique_name_9) for [] in 0)
    }
}

0[0]: {0 for name_0 in unique_name_1}
unique_name_2: {
    0: (lambda: name_0 for unique_name_4 in unique_name_5)
    for unique_name_6 in ()
    if name_0
}

[JelleZijlstra]

I found a fix: #132778. It's definitely in "how did anything work" territory; we were adding the annotation scope to ste_children multiple times, one for each individual module-level annotation. I haven't tracked down exactly why that breaks things, but I guess it has something to do with inline_comprehension() being called multiple times.

[JelleZijlstra]

Thanks for the report!