Elide uninformative traceback indicators in `return` and simple assignment statements

[Zac-HD]

The new traceback indicators can be really nice, though at times also quite verbose. #93883/#93994 by @belm0 improved this situation by skipping the indicators for lines where the entire line was indicated, which helps substantially. I'd like to propose that we take this a small step further:

• Skip indicators for lines with a return statement, where every part execpt the return keyword is indicated; e.g. return foo()
• Skip indicators for lines with a simple assignment statement, where the entire rhs is indicated and the lhs consists of a simple name; e.g. name = some.more(complicated, call=here)

These heuristics are slightly more complicated than "don't indicate the entire line", but I argue that in each of these cases the indicators add little to no information, while compressing the traceback makes it easier to navigate and draws attention to the remaining more-specific indicators.

My motivating example is the traceback reported in pytest-dev/pytest#10466, where I count seven already-elided indicator lines, twelve that would be elided by the return heuristic, seven by the simple-assignment heuristic, and four other lines where the indicators would not be affected by this proposal. I'd even argue that dropping uninformative indicators from a majority (19/30) of traceback lines could be considered a bugfix - indicating on 4/30 lines is quite different to 23/30!

Linked PRs

• gh-99180: Remove traceback anchors in return and assign statements that cover all the displayed range #112670
• gh-99180: Make StackSummary.should_show_carets private #119554
• [3.13] gh-99180: Make StackSummary.should_show_carets private (GH-119554) #119556

[iritkatriel]

CC @pablogsal

[pablogsal]

Thanks a lot for raising these @Zac-HD! I think both of this make sense.

I am a bit wary of complicating the already-complex machinery to detect and deal with the ranges even more because the code is becoming a bit complex to maintain but I can give it a go to see how it looks. Also, we have a bunch of in-flight PRs for this area already so we need to land them before attempting anything here.

I have been thinking about all of this and I think the long term solution may be to use colors if possible instead of underlining the ranges with extra characters. This way we save lines and reduce clutter. If colors would be used in the situations you highlight would this have been acceptable?

[Zac-HD]

I think colors would be overridden by the pytest error machinery, as well as much of our logging infra at work, so that would be equivalent to disabling range indicators entirely. That would certainly solve the problem of uninformative indicators, but at the cost of losing useful information too!

[pablogsal]

Ok, I have this a go and here are my initial thoughts.

1. Is quite annoying that we need to parse again all displayed lines. We currently only parse the relevant parts so in return foo(...) we only parse foo(...). This forces us to ALSO parse all the displayed lines
2. The rule for "Skip indicators for lines with a return statement, where every part except the return keyword is indicated" is not that easy for the following reasons:

   • In return f1("x")("y") if what fails is the second call we currently show:

     return f1("x")("y")
            ~~~~~~~^^^^^
     

     and I don't think we want to remove that because even if you can deduce it from the context (if only the first call fails the whole line will not highlight) I think it adds clarity and I don't want this to be gone.

3. We are currently highlighting with two chars even simple calls so is not that easy to discriminate just based on the number of special anchors.
4. Same thing applies to simple assignments (they are the same case in reality).

[pablogsal]

In any case I created #112670 in case other core devs think otherwise.

@isidentical @lysnikolaou opinions?

[lysnikolaou]

@pablogsal Would it be feasible to only skip the highlights if the highlighted section spans the entire return statement (or the entire rhs) and only one highlighting character is used? This might be a good compromise since it'd still display helpful info when needed (two chars), but the motivating example would still get some benefit from it.

[pablogsal]

@pablogsal Would it be feasible to only skip the highlights if the highlighted section spans the entire return statement (or the entire rhs) and only one highlighting character is used? This might be a good compromise since it'd still display helpful info when needed (two chars), but the motivating example would still get some benefit from it.

Yes, but this still will not cover the OP cases because now we use two characters in many places, including calls. For example:

return baz(1)
       ~~~^^^