seg fault from pynew() in vs code REPL

[johannes-fischer]

Affects: PythonCall

Describe the bug
My vs code integrated REPL crashes if I call PythonCall.pynew(). In a REPL opened in a terminal is works without issues and returns NULL.

Please include the steps required to reproduce the bug. This should include not just code but all the steps required to reproduce the bug

using Pkg
Pkg.activate(temp=True)
Pkg.add("PythonCall")
using PythonCall
PythonCall.pynew()
# However, it doesn't procude an error if the result is not printed by using a semicolon:
PythonCall.pynew();
# But this results in the same seg fault again, even with semicolon:
py = PythonCall.pynew();

Your system
Please provide detailed information about your system:

• The operating system: Ubuntu 20.04
• The version of Julia, Python, PythonCall, JuliaCall and any other affected packages
• If an issue with PythonCall, the output of Base.versioninfo(), Pkg.status() and CondaPkg.status().
• If an issue with JuliaCall, the output of pip list or conda list and juliapkg.status().

Julia Version 1.10.2
Commit bd47eca2c8a (2024-03-01 10:14 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 8 × Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-15.0.7 (ORCJIT, skylake)
Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores)
Environment:
LD_LIBRARY_PATH = /opt/mrtros/lib:/opt/mrtsoftware/local/lib:/opt/mrtsoftware/release/lib
JULIA_EDITOR = code
JULIA_NUM_THREADS =

Status /tmp/jl_ulWM8F/Project.toml
[6099a3de] PythonCall v0.9.20

Additional context
Add any other context about the problem here.

Error message:

julia> PythonCall.pynew()

[19192] signal (11.1): Segmentation fault
in expression starting at none:0
Py_TYPE at /usr/local/src/conda/python-3.11.0/Include/object.h:133 [inlined]
PyObject_GetAttr at /usr/local/src/conda/python-3.11.0/Objects/object.c:906
PyObject_GetAttr at /home/fischer/.julia/packages/PythonCall/S5MOg/src/C/pointers.jl:297 [inlined]
macro expansion at /home/fischer/.julia/packages/PythonCall/S5MOg/src/Core/Py.jl:132 [inlined]
pyhasattr at /home/fischer/.julia/packages/PythonCall/S5MOg/src/Core/builtins.jl:39
pyshow_rule_mimebundle at /home/fischer/.julia/packages/PythonCall/S5MOg/src/Compat/multimedia.jl:30
_jl_invoke at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:3076
pyshowable at /home/fischer/.julia/packages/PythonCall/S5MOg/src/Compat/multimedia.jl:21
showable at /home/fischer/.julia/packages/PythonCall/S5MOg/src/Compat/multimedia.jl:134
unknown function (ip: 0x7fd89f5d5f76)
_jl_invoke at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:3076
showable at ./multimedia.jl:77
display at /home/fischer/.vscode/extensions/julialang.language-julia-1.83.2/scripts/packages/VSCodeServer/src/display.jl:198
unknown function (ip: 0x7fd89f5d5ae9)
_jl_invoke at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:3076
display at ./multimedia.jl:340
unknown function (ip: 0x7fd89f5d5425)
_jl_invoke at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
jl_f__call_latest at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/builtins.c:812
#invokelatest#2 at ./essentials.jl:892 [inlined]
invokelatest at ./essentials.jl:889 [inlined]
print_response at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:315
unknown function (ip: 0x7fd89f5acbde)
#57 at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:284
unknown function (ip: 0x7fd89f5aba95)
_jl_invoke at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:3076
with_repl_linfo at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:569
unknown function (ip: 0x7fd89f589689)
_jl_invoke at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:3076
print_response at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:282
do_respond at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:911
#103 at /home/fischer/.vscode/extensions/julialang.language-julia-1.83.2/scripts/packages/VSCodeServer/src/repl.jl:122
unknown function (ip: 0x7fd89f5a465c)
_jl_invoke at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
jl_f__call_latest at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/builtins.c:812
#invokelatest#2 at ./essentials.jl:892 [inlined]
invokelatest at ./essentials.jl:889 [inlined]
run_interface at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/LineEdit.jl:2656
jfptr_run_interface_90635.1 at /home/fischer/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:3076
run_frontend at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:1312
#62 at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:386
unknown function (ip: 0x7fd89f598792)
_jl_invoke at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
start_task at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/task.c:1238
Allocations: 7472276 (Pool: 7462820; Big: 9456); GC: 12
Segmentation fault
Pane is dead (status 139, Fri Jul 12 14:45:35 2024)

[cjdoris]

Strictly speaking not a bug. What's happening is that pynew() is not safe, there are only a few things you can safely do with the resulting object. Printing at the REPL is safe. However VSCode probably tries to see if the object has a nicer multimedia display implemented - which is unsafe for NULL Python objects.

However it wouldn't be hard to check for NULL in our multimedia code to make this safe. It's reasonable to make showing these objects always be safe.

[johannes-fischer]

Thanks for the explanation! I was particularly irritated by py = PythonCall.pynew(); (suppressing printing of the output with semicolon) also results in a segfault. But you are probably still right, that VS code also tries some unsafe things in this case.

[cjdoris]

I have reproduced the issue and fixed it in the safer-show branch. Can you verify? You can install it like pkg> add PythonCall#safer-show.

I don't understand why the version with the semicolon was crashing, but this seems to fix it. Maybe it's a bug in VSCode, not sure.

[cjdoris]

Ah, I think in the x = PythonCall.pynew(); case, because we have created the variable x, VSCode tries to display it in the workspace viewer.

[johannes-fischer]

Sorry, couldn't get to testing it earlier, but can confirm that this fixes it (tested on current main branch), thanks!