Segmentation fault with threaded FFTW FFTW.set_num_threads(n) and julia --threads=k where k, n > 1

[roflmaostc]

Hey,

I noticed that threaded FFTW.jl (1.0, 1.1, 1.3, 1.4) with Julia (1.6.1, 1.5.4) seems to crash with segmentation faults on my machine (AMD Ryzen 5 5600x). I have no clue what's going on, but it definitely worked in the past (a few weeks/months ago).

The following reproduces it on my system:

julia> FFTW.version # is that the linked library? How to change that?
v"3.3.9"

julia> versioninfo()
Julia Version 1.6.1
Commit 6aaedecc44 (2021-04-23 05:59 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: AMD Ryzen 5 5600X 6-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, generic)
Environment:
  JULIA_NUM_THREADS = 12
  JULIA_COPY_STACKS = yes

➜  julia --threads=1
using Revise, OhMyREPL
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.1 (2021-04-23)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> begin
                      using FFTW
                      FFTW.set_num_threads(2)
                      x = randn((400, 400, 13))
                      fft(x);
              end;

julia> begin
                      using FFTW
                      FFTW.set_num_threads(2)
                      x = randn((400, 400, 13))
                      fft(x);
              end;

julia> begin
                      using FFTW
                      FFTW.set_num_threads(2)
                      x = randn((400, 400, 13))
                      fft(x);
              end;

julia> begin
                      using FFTW
                      FFTW.set_num_threads(2)
                      x = randn((400, 400, 13))
                      fft(x);
              end;

julia> 
➜  julia --threads=2
using Revise, OhMyREPL
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.1 (2021-04-23)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> begin
                      using FFTW
                      FFTW.set_num_threads(2)
                      x = randn((400, 400, 13))
                      fft(x);
              end;

signal (11): Segmentation fault
in expression starting at REPL[1]:1

signal (11): Segmentation fault
in expression starting at REPL[1]:1
spawn_apply at /home/fxw/.julia/artifacts/81791030d1dcd08bf0c67b3e8224cb573d0f5a0a/lib/libfftw3.so (unknown line)
#2 at ./threadingconstructs.jl:169
unknown function (ip: 0x7fec7c04dabf)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
spawn_apply at /home/fxw/.julia/artifacts/81791030d1dcd08bf0c67b3e8224cb573d0f5a0a/lib/libfftw3.so (unknown line)
#2 at ./threadingconstructs.jl:169
unknown function (ip: 0x7fec7c04dabf)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1703 [inlined]
start_task at /buildworker/worker/package_linux64/build/src/task.c:839
jl_init_root_task at /buildworker/worker/package_linux64/build/src/task.c:1288
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1703 [inlined]
start_task at /buildworker/worker/package_linux64/build/src/task.c:839
_julia_init at /buildworker/worker/package_linux64/build/src/init.c:800
repl_entrypoint at /buildworker/worker/package_linux64/build/src/jlapi.c:702
main at julia (unknown line)
jl_init_root_task at /buildworker/worker/package_linux64/build/src/task.c:1288
jl_threadfun at /buildworker/worker/package_linux64/build/src/partr.c:264
start_thread at /lib/x86_64-linux-gnu/libpthread.so.0 (unknown line)
clone at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
Allocations: 5344071 (Pool: 5342120; Big: 1951); GC: 8
[1]    74566 segmentation fault (core dumped)  julia --threads=2

Any idea what could go wrong? I mean it seems not to be very related to the julia version and FFTW.jl version.

Thanks,

Felix

Edit: On my laptop (Intel i7)with same versions, it seems to work fine

[roflmaostc]

It might be due to JULIA_COPY_STACK that was set because of JavaCall.jl.
I don't know why that happens but I close it here and try further digging. Sorry for the trouble.

➜  ~ JULIA_COPY_STACKS="yes"
➜  ~ julia --threads=2      
using Revise, OhMyREPL
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.1 (2021-04-23)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> begin
                             using FFTW
                             FFTW.set_num_threads(12)
                             x = randn((400, 400, 13))
                             fft(x);
                     end;

signal (11): Segmentation fault
in expression starting at REPL[1]:1

signal (11): Segmentation fault
in expression starting at REPL[1]:1
unknown function (ip: 0x7f427973f3cc)
#2 at ./threadingconstructs.jl:169
unknown function (ip: 0x7f4279bc5f8f)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
spawn_apply at /home/fxw/.julia/artifacts/81791030d1dcd08bf0c67b3e8224cb573d0f5a0a/lib/libfftw3.so (unknown line)
#2 at ./threadingconstructs.jl:169
unknown function (ip: 0x7f4279bc5f8f)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1703 [inlined]
start_task at /buildworker/worker/package_linux64/build/src/task.c:839
jl_init_root_task at /buildworker/worker/package_linux64/build/src/task.c:1288
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1703 [inlined]
start_task at /buildworker/worker/package_linux64/build/src/task.c:839
_julia_init at /buildworker/worker/package_linux64/build/src/init.c:800
repl_entrypoint at /buildworker/worker/package_linux64/build/src/jlapi.c:702
main at julia (unknown line)
jl_init_root_task at /buildworker/worker/package_linux64/build/src/task.c:1288
jl_threadfun at /buildworker/worker/package_linux64/build/src/partr.c:264
start_thread at /lib/x86_64-linux-gnu/libpthread.so.0 (unknown line)
clone at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
Allocations: 5200789 (Pool: 5198961; Big: 1828); GC: 7
[1]    3052 segmentation fault (core dumped)  julia --threads=2
➜  ~ JULIA_COPY_STACKS="no" 
➜  ~ julia --threads=2     
using Revise, OhMyREPL
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.1 (2021-04-23)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> begin
                             using FFTW
                             FFTW.set_num_threads(12)
                             x = randn((400, 400, 13))
                             fft(x);
                     end;

julia> begin
                             using FFTW
                             FFTW.set_num_threads(12)
                             x = randn((400, 400, 13))
                             fft(x);
                     end;