Is PythonCall.jl thread safe?

[nrontsis]

When having JULIA_NUM_THREADS>1 and call jax from Julia with PythonCall.jl I occasionally get:

2023-02-23 20:56:02.532960: F external/org_tensorflow/tensorflow/compiler/xla/python/py_array.cc:392] Check failed: PyGILState_Check() 

signal (6): Aborted
in expression starting at [REDACTED]
pthread_kill at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
raise at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_ZN3tsl8internal15LogMessageFatalD1Ev at /home/circleci/.pyenv/versions/3.11.2/lib/python3.11/site-packages/jaxlib/xla_extension.so (unknown line)
_ZN3xla15PyArray_StorageD2Ev.cold at /home/circleci/.pyenv/versions/3.11.2/lib/python3.11/site-packages/jaxlib/xla_extension.so (unknown line)
PyArray_tp_dealloc at /home/circleci/.pyenv/versions/3.11.2/lib/python3.11/site-packages/jaxlib/xla_extension.so (unknown line)
Py_DecRef at /home/circleci/.julia/packages/PythonCall/ZzOaq/src/cpython/pointers.jl:299 [inlined]
#3 at /home/circleci/.julia/packages/PythonCall/ZzOaq/src/gc.jl:57 [inlined]
with_gil at /home/circleci/.julia/packages/PythonCall/ZzOaq/src/cpython/gil.jl:10 [inlined]
enqueue at /home/circleci/.julia/packages/PythonCall/ZzOaq/src/gc.jl:56
py_finalizer at /home/circleci/.julia/packages/PythonCall/ZzOaq/src/Py.jl:46
unknown function (ip: 0x7f1908135c92)
_jl_invoke at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
run_finalizer at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gc.c:280
jl_gc_run_finalizers_in_list at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gc.c:367
run_finalizers at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gc.c:410
jl_gc_run_pending_finalizers at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gc.c:423
jl_mutex_unlock at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/julia_locks.h:131 [inlined]
ijl_task_get_next at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/partr.c:569
poptask at ./task.jl:921
wait at ./task.jl:930
task_done_hook at ./task.jl:634
unknown function (ip: 0x7f19383d2bb2)
_jl_invoke at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_finish_task at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/task.c:254
start_task at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/task.c:942
Allocations: 188027855 (Pool: 187968979; Big: 58876); GC: 57

Received "aborted" signal

[cjdoris]

I've just added an entry to the FAQ: https://github.com/cjdoris/PythonCall.jl/blob/main/docs/src/faq.md#is-pythoncalljuliacall-thread-safe

Feel free to re-open if that doesn't help.

[nrontsis]

Thanks for adding the entry into github's FAQ.

Perhaps that's off-topic, but the FAQ you updated is not up-to-date with the one of the documentation.

[cjdoris]

That's just because I haven't made a release yet. It's in the dev docs.