`breakpoint` does not enter pdb until the "next event" happens

[gaogaotiantian]

Bug report

Bug description:

Consider the following code:

try:
    raise ValueError()
except Exception as exc:
    breakpoint()

You can't stop in the except block, you'll get:

--Return--
> /home/gaogaotiantian/programs/mycpython/example.py(4)<module>()->None
-> breakpoint()
(Pdb)

which is pretty misleading because you can't access anything in the except block:

(Pdb) import sys
(Pdb) sys.exc_info()
(None, None, None)
(Pdb) p exc
*** NameError: name 'exc' is not defined
(Pdb) 

If you put a pass in the main function:

try:
    raise ValueError()
except Exception as exc:
    breakpoint()
pass

At least the result is less misleading. Even though it's still not what I want - I want to stop in the except block.

> /home/gaogaotiantian/programs/mycpython/example.py(5)<module>()
-> pass

The "correct" way to do it is to put another line in the block

try:
    raise ValueError()
except Exception as exc:
    breakpoint()
    pass

We can stop at the breakpoint() actually, but that's a breaking behavior, I don't think we want that. So maybe we should at least document this in pdb so users get less confused?

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-111744: Support opcode events in bdb #111834
• gh-111744: Make breakpoint() enter the debugger immediately #118579

[brandtbucher]

Yes! This behavior remains incredibly surprising to me, and is something I have to re-learn every couple of months. I think I even remember it tripping up @markshannon on at least one occasion, too.

We can stop at the breakpoint() actually, but that's a breaking behavior, I don't think we want that.

Honestly, I find the current behavior annoying and unintuitive enough that a change would be welcome (others may disagree, though). I doubt that there are many "load-bearing" breakpoint() calls in production code. ;)

When I put a breakpoint() call in the code, I always mean to have a breakpoint on that line, not the next one. Doubly so for one-off exception handlers I add for debugging purposes.

What mechanism are you imagining using to stop at the breakpoint line? A one-off C_RETURN event from the breakpoint() call?

[gaogaotiantian]

I believe we need to make breakpoint behave as the same as pdb.set_trace() so C_RETURN won't work. We can set a temp opcode event on the frame that calls into pdb, and catch that. There will be at least a POP_TOP after the call to throw away the return value, and that's before the next line. More importantly, for any code that actually makes a difference, there will be more opcodes (for example, at the end of the except block, temporary local variables after as will be cleared). We can stop before that.

I believe that can be done (without a working prototype yet), but we need to change basically ALL the pdb tests (the doctest ones) because the stop line will change.

I agree this is undoubtedly the expected behavior - the question is whether it's better enough to make the change.

[jaraco]

By pure coincidence, I stumbled onto this issue today in pytest-dev/pytest#11512 (comment).

the question is whether it's better enough to make the change.

My instinct is yes. This surprising behavior has bit me before and added confusion to an already challenging scenario. I'd prefer to meet the user's expectation (that PDB should break in the block where the breakpoint is written) rather than document the deficiency (that PDB breaks before the next statement or return).

On the other hand, when I think about breakpoints defined visually (such as in an IDE) or programmatically (such as through pdb's break command), the user is required to create a statement before which the break will occur. In fact, I don't think it's possible to programmatically break on an implied return, such as happens in the example above. From that perspective, perhaps it makes sense to document to the user that when using breakpoint(), they must ensure that a statement follows it and it's that statement on which the break will occur.

I still lean toward causing the break to occur on the line with the breakpoint call.

[gaogaotiantian]

It's true that the user can only set breakpoints on existing statements, whether through the debugger command line interface or the IDE, but breakpoint() itself is a statement, and I believe there are other similar intrinsics in some compilers like https://learn.microsoft.com/en-us/cpp/intrinsics/debugbreak?view=msvc-170 (even though C is kind of different).

Personally, I don't think how a breakpoint behaves without the statement should impact how it behaves with the explicit statement. However, I do have backward compatibility concerns and we should probably get a broader audience for the feature.