the test suite segfaults on master

[aviatesk]

It successfully ran 12h ago (Julia Version 1.8.0-DEV.527 https://github.com/JuliaDebug/Cthulhu.jl/runs/3603151848), but with the latest master (Version 1.8.0-DEV.542) it yields the following error:

     Testing Cthulhu
Status `/private/var/folders/28/jpsyzp6971s99n6bd4_k6fgm0000gn/T/jl_J9vRyf/Project.toml`
 [da1fd8a2] CodeTracking v1.0.6
 [f68482b8] Cthulhu v2.3.1 `~/julia/packages/Cthulhu`
 [ab62b9b5] DeepDiffs v1.2.0
 [1eca21be] FoldingTrees v1.0.1
 [dc46b164] PerformanceTestTools v0.1.3
 [295af30f] Revise v3.1.19
 [90137ffa] StaticArrays v1.2.12
 [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils`
 [3fa0cd96] REPL `@stdlib/REPL`
 [9a3f8284] Random `@stdlib/Random`
 [8dfed614] Test `@stdlib/Test`
 [cf7118a7] UUIDs `@stdlib/UUIDs`
 [4ec0a83e] Unicode `@stdlib/Unicode`
Status `/private/var/folders/28/jpsyzp6971s99n6bd4_k6fgm0000gn/T/jl_J9vRyf/Manifest.toml`
 [1520ce14] AbstractTrees v0.3.4
 [da1fd8a2] CodeTracking v1.0.6
 [f68482b8] Cthulhu v2.3.1 `~/julia/packages/Cthulhu`
 [ab62b9b5] DeepDiffs v1.2.0
 [1eca21be] FoldingTrees v1.0.1
 [aa1ae85d] JuliaInterpreter v0.8.21
 [6f1432cf] LoweredCodeUtils v2.1.2
 [bac558e1] OrderedCollections v1.4.1
 [dc46b164] PerformanceTestTools v0.1.3
 [ae029012] Requires v1.1.3
 [295af30f] Revise v3.1.19
 [90137ffa] StaticArrays v1.2.12
 [0dad84c5] ArgTools v1.1.1 `@stdlib/ArgTools`
 [56f22d72] Artifacts `@stdlib/Artifacts`
 [2a0f44e3] Base64 `@stdlib/Base64`
 [ade2ca70] Dates `@stdlib/Dates`
 [8ba89e20] Distributed `@stdlib/Distributed`
 [f43a241f] Downloads v1.5.1 `@stdlib/Downloads`
 [7b1f6079] FileWatching `@stdlib/FileWatching`
 [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils`
 [b27032c2] LibCURL v0.6.3 `@stdlib/LibCURL`
 [76f85450] LibGit2 `@stdlib/LibGit2`
 [8f399da3] Libdl `@stdlib/Libdl`
 [37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra`
 [56ddb016] Logging `@stdlib/Logging`
 [d6f4376e] Markdown `@stdlib/Markdown`
 [ca575930] NetworkOptions v1.2.0 `@stdlib/NetworkOptions`
 [44cfe95a] Pkg v1.8.0 `@stdlib/Pkg`
 [de0858da] Printf `@stdlib/Printf`
 [3fa0cd96] REPL `@stdlib/REPL`
 [9a3f8284] Random `@stdlib/Random`
 [ea8e919c] SHA `@stdlib/SHA`
 [9e88b42a] Serialization `@stdlib/Serialization`
 [6462fe0b] Sockets `@stdlib/Sockets`
 [2f01184e] SparseArrays `@stdlib/SparseArrays`
 [10745b16] Statistics `@stdlib/Statistics`
 [fa267f1f] TOML v1.0.0 `@stdlib/TOML`
 [a4e569a6] Tar v1.10.0 `@stdlib/Tar`
 [8dfed614] Test `@stdlib/Test`
 [cf7118a7] UUIDs `@stdlib/UUIDs`
 [4ec0a83e] Unicode `@stdlib/Unicode`
 [e66e0078] CompilerSupportLibraries_jll v0.5.0+0 `@stdlib/CompilerSupportLibraries_jll`
 [deac9b47] LibCURL_jll v7.73.0+4 `@stdlib/LibCURL_jll`
 [29816b5a] LibSSH2_jll v1.9.1+2 `@stdlib/LibSSH2_jll`
 [c8ffd9c3] MbedTLS_jll v2.24.0+2 `@stdlib/MbedTLS_jll`
 [14a3606d] MozillaCACerts_jll v2020.7.22 `@stdlib/MozillaCACerts_jll`
 [4536629a] OpenBLAS_jll v0.3.13+7 `@stdlib/OpenBLAS_jll`
 [83775a58] Zlib_jll v1.2.12+1 `@stdlib/Zlib_jll`
 [8e850b90] libblastrampoline_jll v3.1.0+0 `@stdlib/libblastrampoline_jll`
 [8e850ede] nghttp2_jll v1.41.0+1 `@stdlib/nghttp2_jll`
 [3f19e933] p7zip_jll v16.2.1+1 `@stdlib/p7zip_jll`
     Testing Running tests...
Test Summary: | Pass  Broken  Total
Callsites     |    7       1      8
Test Summary: | Pass  Broken  Total
Expr heads    |    4       1      5
Test Summary:    | Pass  Total
wrapped callinfo |    4      4
Test Summary:                          | Pass  Total
union-split constant-prop'ed callsites |   12     12
Test Summary: | Pass  Total
invoke        |    3      3
Test Summary:        | Pass  Total
Varargs and printing |   35     35
Test Summary: | Pass  Total
MaybeUndef    |    4      4
Test Summary:      | Pass  Total
warntype variables |    1      1
Test Summary:              | Pass  Total
Limit printing (issue #94) |    3      3
Test Summary: | Pass  Total
Issue #132    |    1      1
Test Summary:          | Pass  Total
backedges and treelist |   16     16
Test Summary: | Pass  Total
ascend        |    7      7
Test Summary:    | Pass  Total
ascend interface |    1      1
Test Summary: | Pass  Total
printer test  |   48     48
┌ Info: Running IR test in a subprocess...
│   cmd = `/Users/aviatesk/julia/julia/usr/bin/julia -Cnative -J/Users/aviatesk/julia/julia/usr/lib/julia/sys.dylib --depwarn=yes --check-bounds=yes -g1 --color=yes --startup-file=no --startup-file=no --color=yes --check-bounds=no --code-coverage=none --inline=yes`
└   script = "/Users/aviatesk/julia/packages/Cthulhu/test/irshow.jl"

signal (11): Segmentation fault: 11
in expression starting at /Users/aviatesk/julia/packages/Cthulhu/test/irshow.jl:17
jl_object_id__cold at /Users/aviatesk/julia/julia/src/builtins.c:400
type_hash at /Users/aviatesk/julia/julia/src/jltypes.c:1127
typekey_hash at /Users/aviatesk/julia/julia/src/jltypes.c:1139
jl_precompute_memoized_dt at /Users/aviatesk/julia/julia/src/jltypes.c:1199
inst_datatype_inner at /Users/aviatesk/julia/julia/src/jltypes.c:1512
jl_inst_arg_tuple_type at /Users/aviatesk/julia/julia/src/jltypes.c:1608
arg_type_tuple at /Users/aviatesk/julia/julia/src/gf.c:1845 [inlined]
jl_lookup_generic_ at /Users/aviatesk/julia/julia/src/gf.c:2373 [inlined]
jl_apply_generic at /Users/aviatesk/julia/julia/src/gf.c:2425
jl_apply at /Users/aviatesk/julia/julia/src/./julia.h:1774 [inlined]
do_call at /Users/aviatesk/julia/julia/src/interpreter.c:126
eval_stmt_value at /Users/aviatesk/julia/julia/src/interpreter.c:166 [inlined]
eval_body at /Users/aviatesk/julia/julia/src/interpreter.c:583
eval_body at /Users/aviatesk/julia/julia/src/interpreter.c:516
eval_body at /Users/aviatesk/julia/julia/src/interpreter.c:516
eval_body at /Users/aviatesk/julia/julia/src/interpreter.c:516
eval_body at /Users/aviatesk/julia/julia/src/interpreter.c:516
jl_interpret_toplevel_thunk at /Users/aviatesk/julia/julia/src/interpreter.c:731
jl_toplevel_eval_flex at /Users/aviatesk/julia/julia/src/toplevel.c:889
jl_toplevel_eval_flex at /Users/aviatesk/julia/julia/src/toplevel.c:833
jl_toplevel_eval at /Users/aviatesk/julia/julia/src/toplevel.c:898 [inlined]
jl_toplevel_eval_in at /Users/aviatesk/julia/julia/src/toplevel.c:948
eval at ./boot.jl:368 [inlined]
include_string at ./loading.jl:1207
_jl_invoke at /Users/aviatesk/julia/julia/src/gf.c:0 [inlined]
jl_apply_generic at /Users/aviatesk/julia/julia/src/gf.c:2429
_include at ./loading.jl:1264
include at ./client.jl:451
_jl_invoke at /Users/aviatesk/julia/julia/src/gf.c:0 [inlined]
jl_apply_generic at /Users/aviatesk/julia/julia/src/gf.c:2429
jl_apply at /Users/aviatesk/julia/julia/src/./julia.h:1774 [inlined]
do_call at /Users/aviatesk/julia/julia/src/interpreter.c:126
eval_stmt_value at /Users/aviatesk/julia/julia/src/interpreter.c:166 [inlined]
eval_body at /Users/aviatesk/julia/julia/src/interpreter.c:583
jl_interpret_toplevel_thunk at /Users/aviatesk/julia/julia/src/interpreter.c:731
jl_toplevel_eval_flex at /Users/aviatesk/julia/julia/src/toplevel.c:889
jl_toplevel_eval_flex at /Users/aviatesk/julia/julia/src/toplevel.c:833
jl_toplevel_eval at /Users/aviatesk/julia/julia/src/toplevel.c:898 [inlined]
jl_toplevel_eval_in at /Users/aviatesk/julia/julia/src/toplevel.c:948
eval at ./boot.jl:368 [inlined]
exec_options at ./client.jl:268
_start at ./client.jl:495
jfptr__start_36217 at /Users/aviatesk/julia/julia/usr/lib/julia/sys.dylib (unknown line)
_jl_invoke at /Users/aviatesk/julia/julia/src/gf.c:0 [inlined]
jl_apply_generic at /Users/aviatesk/julia/julia/src/gf.c:2429
jl_apply at /Users/aviatesk/julia/julia/src/./julia.h:1774 [inlined]
true_main at /Users/aviatesk/julia/julia/src/jlapi.c:548
jl_repl_entrypoint at /Users/aviatesk/julia/julia/src/jlapi.c:690
Allocations: 729523 (Pool: 729061; Big: 462); GC: 1
Test Failed at /Users/aviatesk/.julia/packages/PerformanceTestTools/cYTSX/src/PerformanceTestTools.jl:34
  Expression: success(pipeline(`$cmd -e $code`; stdout = stdout, stderr = stderr))
ERROR: LoadError: There was an error during testing
in expression starting at /Users/aviatesk/julia/packages/Cthulhu/test/codeview.jl:42
in expression starting at /Users/aviatesk/julia/packages/Cthulhu/test/runtests.jl:643
ERROR: Package Cthulhu errored during testing
Stacktrace:
 [1] pkgerror(msg::String)
   @ Pkg.Types ~/julia/julia/usr/share/julia/stdlib/v1.8/Pkg/src/Types.jl:67
 [2] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, julia_args::Cmd, test_args::Cmd, test_fn::Nothing, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool)
   @ Pkg.Operations ~/julia/julia/usr/share/julia/stdlib/v1.8/Pkg/src/Operations.jl:1687
 [3] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, test_fn::Nothing, julia_args::Cmd, test_args::Cmd, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool, kwargs::Base.Pairs{Symbol, Base.TTY, Tuple{Symbol}, NamedTuple{(:io,), Tuple{Base.TTY}}})
   @ Pkg.API ~/julia/julia/usr/share/julia/stdlib/v1.8/Pkg/src/API.jl:426
 [4] test(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ Pkg.API ~/julia/julia/usr/share/julia/stdlib/v1.8/Pkg/src/API.jl:154
 [5] test(pkgs::Vector{Pkg.Types.PackageSpec})
   @ Pkg.API ~/julia/julia/usr/share/julia/stdlib/v1.8/Pkg/src/API.jl:144
 [6] test(; name::Nothing, uuid::Nothing, version::Nothing, url::Nothing, rev::Nothing, path::Nothing, mode::Pkg.Types.PackageMode, subdir::Nothing, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ Pkg.API ~/julia/julia/usr/share/julia/stdlib/v1.8/Pkg/src/API.jl:169
 [7] test()
   @ Pkg.API ~/julia/julia/usr/share/julia/stdlib/v1.8/Pkg/src/API.jl:163
 [8] top-level scope
   @ none:1

It seems that PerformanceTestTools.@include("irshow.jl") causes this error, and certainly we can get the same error by running the script directly:

❯ ~/julia/julia/usr/bin/julia irshow.jl

signal (11): Segmentation fault: 11
in expression starting at /Users/aviatesk/julia/packages/Cthulhu/test/irshow.jl:17
jl_object_id__cold at /Users/aviatesk/julia/julia/src/builtins.c:400
type_hash at /Users/aviatesk/julia/julia/src/jltypes.c:1127
typekey_hash at /Users/aviatesk/julia/julia/src/jltypes.c:1139
jl_precompute_memoized_dt at /Users/aviatesk/julia/julia/src/jltypes.c:1199
inst_datatype_inner at /Users/aviatesk/julia/julia/src/jltypes.c:1512
jl_inst_arg_tuple_type at /Users/aviatesk/julia/julia/src/jltypes.c:1608
arg_type_tuple at /Users/aviatesk/julia/julia/src/gf.c:1845 [inlined]
jl_lookup_generic_ at /Users/aviatesk/julia/julia/src/gf.c:2373 [inlined]
jl_apply_generic at /Users/aviatesk/julia/julia/src/gf.c:2425
jl_apply at /Users/aviatesk/julia/julia/src/./julia.h:1774 [inlined]
do_call at /Users/aviatesk/julia/julia/src/interpreter.c:126
eval_stmt_value at /Users/aviatesk/julia/julia/src/interpreter.c:166 [inlined]
eval_body at /Users/aviatesk/julia/julia/src/interpreter.c:583
eval_body at /Users/aviatesk/julia/julia/src/interpreter.c:516
eval_body at /Users/aviatesk/julia/julia/src/interpreter.c:516
eval_body at /Users/aviatesk/julia/julia/src/interpreter.c:516
eval_body at /Users/aviatesk/julia/julia/src/interpreter.c:516
jl_interpret_toplevel_thunk at /Users/aviatesk/julia/julia/src/interpreter.c:731
jl_toplevel_eval_flex at /Users/aviatesk/julia/julia/src/toplevel.c:889
jl_toplevel_eval_flex at /Users/aviatesk/julia/julia/src/toplevel.c:833
jl_toplevel_eval at /Users/aviatesk/julia/julia/src/toplevel.c:898 [inlined]
jl_toplevel_eval_in at /Users/aviatesk/julia/julia/src/toplevel.c:948
eval at ./boot.jl:368 [inlined]
include_string at ./loading.jl:1207
_jl_invoke at /Users/aviatesk/julia/julia/src/gf.c:0 [inlined]
jl_apply_generic at /Users/aviatesk/julia/julia/src/gf.c:2429
_include at ./loading.jl:1264
include at ./Base.jl:416
_jl_invoke at /Users/aviatesk/julia/julia/src/gf.c:0 [inlined]
jl_apply_generic at /Users/aviatesk/julia/julia/src/gf.c:2429
exec_options at ./client.jl:292
_start at ./client.jl:495
jfptr__start_36217 at /Users/aviatesk/julia/julia/usr/lib/julia/sys.dylib (unknown line)
_jl_invoke at /Users/aviatesk/julia/julia/src/gf.c:0 [inlined]
jl_apply_generic at /Users/aviatesk/julia/julia/src/gf.c:2429
jl_apply at /Users/aviatesk/julia/julia/src/./julia.h:1774 [inlined]
true_main at /Users/aviatesk/julia/julia/src/jlapi.c:548
jl_repl_entrypoint at /Users/aviatesk/julia/julia/src/jlapi.c:690
Allocations: 729408 (Pool: 728944; Big: 464); GC: 1

[simeonschaub]

Seems to be caused by JuliaLang/julia#42172. I will take a look

[simeonschaub]

Ah, I was wrong. The segfaults are actually nondeterministic, so I was able to observe this on JuliaLang/julia@2f47dae (1.8.0-DEV.527) as well.

[simeonschaub]

This is really weird... I can reproduce this on 1.7, just not with the official 1.7-beta3 build (also built exactly the same commit myself and got the segfault). I also deleted .julia/compiled in case there might be some corruption going on from previous attempts, but that didn't help.

The source of the error seems to be that the datatype of the argument f in the call to process from test/utils.jl is not properly initialized and just contains a bunch of garbage. I am a bit out of ideas here, any ideas why that could be?

[aviatesk]

Somehow this diff works for me...

diff --git a/test/irshow.jl b/test/irshow.jl
index 55a493c..54c1a84 100644
--- a/test/irshow.jl
+++ b/test/irshow.jl
@@ -2,7 +2,8 @@ using Cthulhu, Test, DeepDiffs
 
 include("utils.jl")
 
-foo = Core.eval(Module(), Meta.parseall(raw"""
+m = Module()
+Core.eval(m, Meta.parseall(raw"""
 function foo(x, y)
     z = x + y
     if z < 4
@@ -18,7 +19,7 @@ end
     tf = (true, false)
 
     @testset "optimize: $optimize" for optimize in tf
-        _, src, infos, mi, rt, slottypes = process(foo, (Int, Int); optimize);
+        _, src, infos, mi, rt, slottypes = process(m.foo, (Int, Int); optimize);
 
         @testset "debuginfo: $debuginfo" for debuginfo in instances(Cthulhu.DInfo.DebugInfo)
             @testset "iswarn: $iswarn" for iswarn in tf