Doesn't work with -O3

[PallHaraldsson]

It's probably a fluke but I thought you would like to know. And I've not seen this on defaults, and only on 1.9.0-rc1

$ julia -O3

julia> @time using PythonCall
[ Info: Precompiling PythonCall [6099a3de-0909-46bc-b1f4-468b9a2dfc0d]
free(): invalid pointer

[34577] signal (6.-6): Aborted
in expression starting at REPL[4]:1
pthread_kill at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
raise at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x7ff83ef1d6f5)
unknown function (ip: 0x7ff83ef34d7b)
unknown function (ip: 0x7ff83ef36ac3)
free at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_PyMem_RawFree at /usr/local/src/conda/python-3.10.6/Objects/obmalloc.c:127 [inlined]
PyMem_RawFree at /usr/local/src/conda/python-3.10.6/Objects/obmalloc.c:595 [inlined]
_PyObject_Free at /usr/local/src/conda/python-3.10.6/Objects/obmalloc.c:2248 [inlined]
_PyObject_Free at /usr/local/src/conda/python-3.10.6/Objects/obmalloc.c:2239 [inlined]
PyObject_Free at /usr/local/src/conda/python-3.10.6/Objects/obmalloc.c:709 [inlined]
object_dealloc at /usr/local/src/conda/python-3.10.6/Objects/typeobject.c:4510
Py_DECREF at /usr/local/src/conda/python-3.11.3/Include/object.h:538 [inlined]
Py_XDECREF at /usr/local/src/conda/python-3.11.3/Include/object.h:602 [inlined]
_PyImport_LoadDynamicModuleWithSpec at /usr/local/src/conda/python-3.11.3/Python/importdl.c:246 [inlined]
_imp_create_dynamic_impl at /usr/local/src/conda/python-3.11.3/Python/import.c:2385 [inlined]
_imp_create_dynamic at /usr/local/src/conda/python-3.11.3/Python/clinic/import.c.h:446
cfunction_vectorcall_FASTCALL at /usr/local/src/conda/python-3.11.3/Objects/methodobject.c:427
do_call_core at /usr/local/src/conda/python-3.11.3/Python/ceval.c:7328 [inlined]
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.3/Python/ceval.c:5380
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.3/Include/internal/pycore_ceval.h:73 [inlined]
_PyEval_Vector at /usr/local/src/conda/python-3.11.3/Python/ceval.c:6438 [inlined]
_PyFunction_Vectorcall at /usr/local/src/conda/python-3.11.3/Objects/call.c:393
_PyObject_VectorcallTstate at /usr/local/src/conda/python-3.11.3/Include/internal/pycore_call.h:92 [inlined]
object_vacall at /usr/local/src/conda/python-3.11.3/Objects/call.c:819
PyObject_CallMethodObjArgs at /usr/local/src/conda/python-3.11.3/Objects/call.c:879
import_find_and_load at /usr/local/src/conda/python-3.11.3/Python/import.c:1736 [inlined]
PyImport_ImportModuleLevelObject at /usr/local/src/conda/python-3.11.3/Python/import.c:1835
import_name at /usr/local/src/conda/python-3.11.3/Python/ceval.c:7428 [inlined]
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.3/Python/ceval.c:3950
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.3/Include/internal/pycore_ceval.h:73 [inlined]
_PyEval_Vector at /usr/local/src/conda/python-3.11.3/Python/ceval.c:6438
PyEval_EvalCode at /usr/local/src/conda/python-3.11.3/Python/ceval.c:1154
builtin_exec_impl at /usr/local/src/conda/python-3.11.3/Python/bltinmodule.c:1077 [inlined]
builtin_exec at /usr/local/src/conda/python-3.11.3/Python/clinic/bltinmodule.c.h:465
cfunction_vectorcall_FASTCALL_KEYWORDS at /usr/local/src/conda/python-3.11.3/Objects/methodobject.c:443
do_call_core at /usr/local/src/conda/python-3.11.3/Python/ceval.c:7328 [inlined]
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.3/Python/ceval.c:5380
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.3/Include/internal/pycore_ceval.h:73 [inlined]
_PyEval_Vector at /usr/local/src/conda/python-3.11.3/Python/ceval.c:6438 [inlined]
_PyFunction_Vectorcall at /usr/local/src/conda/python-3.11.3/Objects/call.c:393
_PyObject_VectorcallTstate at /usr/local/src/conda/python-3.11.3/Include/internal/pycore_call.h:92 [inlined]
object_vacall at /usr/local/src/conda/python-3.11.3/Objects/call.c:819
PyObject_CallMethodObjArgs at /usr/local/src/conda/python-3.11.3/Objects/call.c:879
import_find_and_load at /usr/local/src/conda/python-3.11.3/Python/import.c:1736 [inlined]
PyImport_ImportModuleLevelObject at /usr/local/src/conda/python-3.11.3/Python/import.c:1835
builtin___import___impl at /usr/local/src/conda/python-3.11.3/Python/bltinmodule.c:277 [inlined]
builtin___import__ at /usr/local/src/conda/python-3.11.3/Python/clinic/bltinmodule.c.h:78
cfunction_vectorcall_FASTCALL_KEYWORDS at /usr/local/src/conda/python-3.11.3/Objects/methodobject.c:443
_PyObject_VectorcallTstate at /usr/local/src/conda/python-3.11.3/Include/internal/pycore_call.h:92 [inlined]
_PyObject_CallFunctionVa at /usr/local/src/conda/python-3.11.3/Objects/call.c:536 [inlined]
PyObject_CallFunction at /usr/local/src/conda/python-3.11.3/Objects/call.c:558
PyImport_Import at /usr/local/src/conda/python-3.11.3/Python/import.c:2024
PyImport_Import at /home/pharaldsson/.julia/packages/PythonCall/dsECZ/src/cpython/pointers.jl:299 [inlined]
macro expansion at /home/pharaldsson/.julia/packages/PythonCall/dsECZ/src/Py.jl:131 [inlined]
pyimport at /home/pharaldsson/.julia/packages/PythonCall/dsECZ/src/concrete/import.jl:11
init_consts at /home/pharaldsson/.julia/packages/PythonCall/dsECZ/src/concrete/consts.jl:189
#199 at /home/pharaldsson/.julia/packages/PythonCall/dsECZ/src/PythonCall.jl:87
with_gil at /home/pharaldsson/.julia/packages/PythonCall/dsECZ/src/cpython/gil.jl:10 [inlined]
with_gil at /home/pharaldsson/.julia/packages/PythonCall/dsECZ/src/cpython/gil.jl:9 [inlined]
__init__ at /home/pharaldsson/.julia/packages/PythonCall/dsECZ/src/PythonCall.jl:86
unknown function (ip: 0x7ff7e959e0ff)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1878 [inlined]
jl_module_run_initializer at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/module.c:982
register_restored_modules at ./loading.jl:1074
_include_from_serialized at ./loading.jl:1020
_tryrequire_from_serialized at ./loading.jl:1401
unknown function (ip: 0x7ff7e955c0fd)
_require at ./loading.jl:1775
_require_prelocked at ./loading.jl:1619
unknown function (ip: 0x7ff7e9554c36)
macro expansion at ./loading.jl:1607 [inlined]
macro expansion at ./lock.jl:267 [inlined]
require at ./loading.jl:1570
unknown function (ip: 0x7ff7e9551c46)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1878 [inlined]
call_require at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:466 [inlined]
eval_import_path at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:503
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:731
top-level scope at ./timing.jl:273 [inlined]
top-level scope at ./REPL[4]:0
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:903
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:856
ijl_toplevel_eval_in at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
eval_user_input at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:153
repl_backend_loop at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:249
#start_repl_backend#46 at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:234
start_repl_backend at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:231
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
#run_repl#59 at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:377
run_repl at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:363
jfptr_run_repl_60577.clone_1 at /home/pharaldsson/.julia/juliaup/julia-1.9.0-rc1+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
#1017 at ./client.jl:421
jfptr_YY.1017_27809.clone_1 at /home/pharaldsson/.julia/juliaup/julia-1.9.0-rc1+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1878 [inlined]
jl_f__call_latest at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/builtins.c:774
#invokelatest#2 at ./essentials.jl:816 [inlined]
invokelatest at ./essentials.jl:813 [inlined]
run_main_repl at ./client.jl:405
exec_options at ./client.jl:322
_start at ./client.jl:522
jfptr__start_33350.clone_1 at /home/pharaldsson/.julia/juliaup/julia-1.9.0-rc1+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1878 [inlined]
true_main at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/jlapi.c:573
jl_repl_entrypoint at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/jlapi.c:717
main at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/cli/loader_exe.c:59
unknown function (ip: 0x7ff83eebdd8f)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x401098)
Allocations: 20040012 (Pool: 20031256; Big: 8756); GC: 23
Aborted (core dumped)

[then right after]
$ julia -O3

julia> @time using PythonCall
  2.313616 seconds (1.43 M allocations: 89.947 MiB, 5.32% gc time, 1.59% compilation time)

Potentially this should be reported at Julialang, before 1.9 release (seems imminent) or after, if also happens there.

[cjdoris]

I'm not seeing this on v1.9 - is it still an issue for you?

[PallHaraldsson]

It's not an issue for me on on 1.9.0 with -O3:

julia> @time using PythonCall
[ Info: Precompiling PythonCall [6099a3de-0909-46bc-b1f4-468b9a2dfc0d]
    CondaPkg Found dependencies: /home/pharaldsson/.julia/packages/PythonCall/dsECZ/CondaPkg.toml
    CondaPkg Dependencies already up to date
 87.749025 seconds (5.42 M allocations: 391.901 MiB, 0.30% gc time, 0.14% compilation time)

Then on default settings (or higher same):

julia> @time using PythonCall
  1.968430 seconds (835.54 k allocations: 55.378 MiB, 5.06% gc time, 1.57% compilation time)

Note, when this happened before if was ok on default settings then only failed on -O3 and I didn't try to compile in that order. I'm not sure it mattered, likely would have with -O3 first too (or not deterministic). I guess it's your say to close this and I support that, you could always reopen, wait and see if happens again.

The timing, just under 2 sec (and the non-worrying but long precompile time) is a little longer than ideal. But that's a separate issue. Maybe the new PrecompileTools.jl will help.

[cjdoris]

Ok I'll close - let me know any future issues though thanks.