exception types not displayed in knitr markdown Rnw document

[brry]

In Rnw Python chunks, exception types are no longer included in the output.
In version 1.28, the example below showed
## ValueError: invalid literal for int() with base 10: 'charstring'
starting with version 1.29 (lots of exception changes), it only shows
## invalid literal for int() with base 10: 'charstring'
without the ValueError: part.

Is this intentional?
Can I get the exception types back? (I use it in slides for teaching Python, so they are really nice to have).

Note: I originally filed the issue in knitr, but it belongs here.

Content of ShowPyException.Rnw file:

---
title: "ShowPyException"
output: beamer_presentation
---

## slide title

```{python, error=TRUE}
obj = "charstring"
int(obj)
```

[t-kalinowski]

Hi @brry, thanks for reporting! This change was not intentional, and we'll certainly want to fix this before the next release.

Would you like to submit a PR?

[brry]

I'd like to, but I don't know where to get started. If you could give me a pointer, I would love to try.

[t-kalinowski]

The work of parsing, evaluating, and processing output from python chunks happens in eng_python(). You may want to start by stepping through that function in an interactive R session (debug(reticulate::eng_python); knitr::knit(...), and note where the error is getting captured and printed. Then add a case that will construct the correct exception message from the captured exception object.

The python code gets evaluated in a tryCatch (

reticulate/R/knitr-engine.R

Line 256 in efacf71

tryCatch(py_compile_eval(snippet, compile_mode), error = identity)

), so that means we'll have access to the live python exception object, which we can use to construct the correct message.
For example:

> e = tryCatch(py_run_string("raise Exception('foo')"), error = identity)
> e
Exception('foo')
> class(e)
[1] "python.builtin.Exception"     "python.builtin.BaseException" "python.builtin.object"       
[4] "error"                        "condition"                   
> dput(e)
<environment>
> str(e)
foo
> py_list_attributes(e)
 [1] "__cause__"            "__class__"            "__context__"          "__delattr__"         
 [5] "__dict__"             "__dir__"              "__doc__"              "__eq__"              
 [9] "__format__"           "__ge__"               "__getattribute__"     "__getstate__"        
[13] "__gt__"               "__hash__"             "__init__"             "__init_subclass__"   
[17] "__le__"               "__lt__"               "__ne__"               "__new__"             
[21] "__reduce__"           "__reduce_ex__"        "__repr__"             "__setattr__"         
[25] "__setstate__"         "__sizeof__"           "__str__"              "__subclasshook__"    
[29] "__suppress_context__" "__traceback__"        "add_note"             "args"                
[33] "call"                 "trace"                "with_traceback"      
> e$`__class__`
<class 'Exception'>
> e$args
[[1]]
[1] "foo"

> py_repr(e)
[1] "Exception('foo')"

To construct the error message, you can directly in the engine (maybe in eng_python_autoprint?) call something like import("traceback")$format_exception(...), or via reticulate::py_last_error()$message, or perhaps use py_repr() (though I'm not 100% sure if py_repr() will be faithful to the exception printing you'd see in a REPL).

[brry]

So far, I've tracked it down to wrap at the end of eng_python.
outputs$data() still has the error name, the result of wrap (knitr::engine_output) lost it. As far as I can tell, outputs$data()[[2]] is the e in the example above and str(e) also loses the name, hence I guess it never gets wrapped.
In case it is knitr::engine_output that needs fixing, I think line 82 points at sew as the likely place.

I will keep investigating once I have some more time.