traceback.walk_stack(None) does not behave the same as traceback.walk_stack(inspect.currentframe())

[VincentVanlaer]

Bug report

traceback.walk_stack(None) omits the calling frame, making it equivalent to traceback.walk_stack(inspect.currentframe().f_back). Given the documentation just says "If f is None, the current stack is used.", I would have expected traceback.walk_stack(None) to be equivalent to traceback.walk_stack(inspect.currentframe()).

Example code

The following code

import traceback
import inspect

def abc():
    print(list(traceback.walk_stack(None)))
    print(list(traceback.walk_stack(inspect.currentframe())))
    print(list(traceback.walk_stack(inspect.currentframe().f_back)))

abc()

has as output

[(<frame at 0x7f66ac559a40, file '/home/vincenttc/data/temp/test/test.py', line 10, code <module>>, 10)]
[(<frame at 0x7f66ac55fa60, file '/home/vincenttc/data/temp/test/test.py', line 7, code abc>, 7), (<frame at 0x7f66ac559a40, file '/home/vincenttc/data/temp/test/test.py', line 10, code <module>>, 10)]
[(<frame at 0x7f66ac559a40, file '/home/vincenttc/data/temp/test/test.py', line 10, code <module>>, 10)]

Note that the frame for abc is missing in the first and last line.

Your environment

• CPython versions tested on: 3.8.10 (Ubuntu 20.04), 3.10.5 (Arch Linux)

• PR: GH-96092: restore frame height of traceback.walk_stack to that of pyt… #99015

• PR: gh-96092: Fix traceback.walk_stack(None) skipping too many frames #129330

[iritkatriel]

On main your code doesn't work at all:

>>> import traceback
>>> import inspect
>>> 
>>> def abc():
...     print(list(traceback.walk_stack(None)))
...     print(list(traceback.walk_stack(inspect.currentframe())))
...     print(list(traceback.walk_stack(inspect.currentframe().f_back)))
... 
>>> abc()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in abc
  File "/Users/iritkatriel/src/cpython/Lib/traceback.py", line 333, in walk_stack
    f = sys._getframe().f_back.f_back.f_back.f_back
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'f_back'

I see that this function changed in

commit 5644c7b3ffd49bed58dc095be6e6148e0bb4431e
Author: Ammar Askar <ammar@ammaraskar.com>
Date:   Sun Jul 4 19:14:33 2021 -0400

    bpo-43950: Print columns in tracebacks (PEP 657) (GH-26958)
    
    The traceback.c and traceback.py mechanisms now utilize the newly added code.co_positions and PyCode_Addr2Location
    to print carets on the specific expressions involved in a traceback.
    
    Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
    Co-authored-by: Ammar Askar <ammar@ammaraskar.com>
    Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>

This part:

@@ -309,7 +321,7 @@ def walk_stack(f):
     current stack is used. Usually used with StackSummary.extract.
     """
     if f is None:
-        f = sys._getframe().f_back.f_back
+        f = sys._getframe().f_back.f_back.f_back.f_back
     while f is not None:
         yield f, f.f_lineno
         f = f.f_back

CC @ammaraskar @pablogsal @isidentical

[pablogsal]

I am currently in release hell, @ammaraskar @isidentical can you take a look?

[ammaraskar]

Added details on the root cause here: #99015 (comment)

It looks like traceback.walk_stack was never really meant to be called without immediately using StackSummary.extract on it. It is a generator so the:

    if f is None:
        f = sys._getframe().f_back.f_back

is a bit misleading, those lines don't run when you call traceback.walk_stack but only when you start iterating over the generator.

So if you do something like:

def outer():
    return inner()
def inner():
    return traceback.walk_stack(None)

frames = outer()


def print_frames(f):
    print(list(f))

print_frames(frames)

You might expect it to print module -> outer -> inner but it would actually get module -> print_frames because that is where the walk_stack generator is consumed.

---

My intuition is we should fix walk_stack to return a generator that iterates over the frames from when it is called, rather than iterated over. While this is a breaking change, it seems like walk_stack(None) is almost never called in the stdlib or tests without an immediate StackSummary.extract nor did I find anything in a quick search over Github.

[graingert]

@ammaraskar we use it here: https://github.com/dask/distributed/blob/c137ac0856af75a7ee54259ff86a15f6be1babe4/distributed/client.py#L2929