Cannot perform frame jump after handled exception

[dkrystki]

For given code:

def fun():
    a = 1  # <---- jump location

    try:
        b = 1 / 0
    except ZeroDivisionError as e:
        pass

    c = 3  # current location


if __name__ == "__main__":
    fun()

Jumping from current location to jump location yields ValueError: can't jump from within an exception handler

Your environment

• CPython versions tested on: 3.11.0a7
• Operating system and architecture: Linux q 5.4.0-42-generic x64

Works as expected for python < 3.11

[serhiy-storchaka]

@markshannon

[markshannon]

@dkrystki Do you have a full reproducer?
You haven't said how you are performing the jump.

[dkrystki]

For the presented example I performed jump using PyCharm.
Not sure if I know how to reproduce it using settrace (I can try) but under the hood PyCharm is simply setting f_lineno to perform a jump.

Edit: For my library purposes I am setting f_lineno manually and it is causing the same issue

[dkrystki]

I managed to make a full reproducer:

import sys

def fun():
    a = 1  # <---- jump location

    try:
        b = 1 / 0
    except ZeroDivisionError as e:
        pass

    c = 3  # current location


def trace(frame, event, arg):
    if event == "line" and frame.f_lineno == 11:
        frame.f_lineno = 4

    return trace


sys.settrace(trace)

if __name__ == "__main__":
    fun()

Traceback:

Traceback (most recent call last):
  File "<cwd>/cakeshop.py", line 24, in <module>
    fun()
    ^^^^^
  File "<cwd>/cakeshop.py", line 11, in fun
    c = 3  # current location
        ^
  File "<cwd>/cakeshop.py", line 11, in fun
    c = 3  # current location
        ^
  File "<cwd>/cakeshop.py", line 16, in trace
    frame.f_lineno = 4
    ^^^^^^^^^^^^^^
ValueError: can't jump from within an exception handler

[serhiy-storchaka]

diff --git a/Lib/test/test_sys_settrace.py b/Lib/test/test_sys_settrace.py
index b1c8f6f80a..34126464bc 100644
--- a/Lib/test/test_sys_settrace.py
+++ b/Lib/test/test_sys_settrace.py
@@ -2217,6 +2217,16 @@ def test_jump_backwards_into_try_except_block(output):
             raise
         output.append(6)
 
+    @jump_test(7, 1, [1, 3, 6, 1, 3, 6])
+    def test_jump_backwards_over_try_except_block(output):
+        output.append(1)
+        try:
+            output.append(3)
+            1/0
+        except ZeroDivisionError:
+            output.append(6)
+        x = 7
+
     # 'except' with a variable creates an implicit finally block
     @jump_test(5, 7, [4], (ValueError, 'within'))
     def test_no_jump_between_except_blocks_2(output):

[serhiy-storchaka]

All works if add anything above or below the assignment.

[sweeneyde]

For what it's worth, this is the problematic result from the reproducer.

Bytecode:

3           0 RESUME                   0

4           2 LOAD_CONST               1 (1)
            4 STORE_FAST               0 (a)

6           6 NOP

7           8 LOAD_CONST               1 (1)
           10 LOAD_CONST               2 (0)
           12 BINARY_OP               11 (/)
           16 STORE_FAST               1 (b)

11          18 LOAD_CONST               3 (3)
           20 STORE_FAST               3 (c)
           22 LOAD_CONST               0 (None)
           24 RETURN_VALUE
      >>   26 PUSH_EXC_INFO

8          28 LOAD_GLOBAL              0 (ZeroDivisionError)
           40 CHECK_EXC_MATCH
           42 POP_JUMP_FORWARD_IF_FALSE    13 (to 70)
           44 STORE_FAST               2 (e)

9          46 POP_EXCEPT
           48 LOAD_CONST               0 (None)
           50 STORE_FAST               2 (e)
           52 DELETE_FAST              2 (e)

11          54 LOAD_CONST               3 (3)
           56 STORE_FAST               3 (c)
           58 LOAD_CONST               0 (None)
           60 RETURN_VALUE
           62 LOAD_CONST               0 (None)
           64 STORE_FAST               2 (e)
           66 DELETE_FAST              2 (e)
           68 RERAISE                  1

8     >>   70 RERAISE                  0
      >>   72 COPY                     3
           74 POP_EXCEPT
           76 RERAISE                  1
ExceptionTable:
8 to 16 -> 26 [0]
26 to 44 -> 72 [1] lasti
62 to 70 -> 72 [1] lasti

Result from mark_stacks():

int64_t *stacks = {
     [0] =  0,  [1] =  0,  [2] =  3,  [3] =  0,  [4] =  0,
     [5] =  3,  [6] = 15,  [7] =  3,  [8] =  3,  [9] =  0,
    [10] =  3, [11] =  0, [12] =  3, [13] = -2, [14] = -2,
    [15] = -2, [16] = -2, [17] = -2, [18] = -2, [19] = -2,
    [20] = -2, [21] = -2, [22] = -2, [23] = -2, [24] = -2,
    [25] = -2, [26] = -2, [27] = -2, [28] = -2, [29] = -2,
    [30] = -2, [31] = -2, [32] = -2, [33] = -2, [34] = -2,
    [35] = -2, [36] = -2, [37] = -2, [38] = -2, [39] = -2,
};

Here, -2 means UNINITIALIZED. This is a result of the following code in mark_stacks:

        for (i = 0; i < len; i++) {
            int64_t next_stack = stacks[i];
            if (next_stack == UNINITIALIZED) {
                continue;
            }
            // ...
        }

After RETURN_VALUE, the rest of the stacks are UNINITIALIZED and remain UNINITIALIZED as they're skipped over. I don't know the correct solution.

BTW, bisected to adcd220

[markshannon]

I'm making this a deferred blocker.
I don't think it needs to block the next beta, but I definitely want to fix it before release

[markshannon]

We need to use the exception handling table to fill in the stacks for exception handlers.
Fiddly, but not too difficult.