Closed
Description
While running fmriprep on the dataset of #703, I've found out the following. At a given point in time, few cpus are being used:
top - 13:59:52 up 61 days, 19:46, 1 user, load average: 68.15, 68.19, 68.17
Tasks: 2708 total, 69 running, 2639 sleeping, 0 stopped, 0 zombie
%Cpu0 : 97.2 us, 2.8 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu1 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu2 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu3 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu4 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu5 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu6 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu7 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu8 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu9 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu10 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu11 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu12 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu13 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu14 : 0.0 us, 0.3 sy, 0.0 ni, 99.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu15 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu16 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu17 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu18 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu19 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu20 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu21 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu22 : 1.2 us, 2.2 sy, 0.0 ni, 96.6 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu23 : 1.3 us, 1.6 sy, 0.0 ni, 97.2 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu24 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu25 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu26 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu27 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu28 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu29 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu30 : 25.5 us, 0.3 sy, 0.0 ni, 74.1 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu31 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu32 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu33 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu34 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu35 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu36 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu37 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu38 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu39 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu40 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu41 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu42 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu43 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu44 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu45 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu46 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu47 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu48 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu49 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu50 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu51 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu52 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu53 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu54 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
For this I configured --n_procs 68. The top command shows a lot of processes being run in parallel though:
top - 14:07:47 up 61 days, 19:54, 1 user, load average: 68.03, 68.12, 68.15
Tasks: 2701 total, 69 running, 2632 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.4 us, 0.1 sy, 0.0 ni, 99.5 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 98696408 total, 73720752 free, 18519412 used, 6456240 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 75978360 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
39496 oesteban 20 0 1563400 925260 20288 S 18.1 0.9 303:21.13 fmriprep
90832 oesteban 20 0 164640 5096 1580 R 6.2 0.0 0:01.97 top
9 root 20 0 0 0 0 S 1.9 0.0 97:20.55 rcu_sched
39601 oesteban 20 0 761848 250376 10368 R 1.6 0.3 2:59.80 fmriprep
89853 oesteban 20 0 79592 50020 10732 R 1.6 0.1 0:24.94 antsApplyTransf
89864 oesteban 20 0 79592 50024 10732 R 1.6 0.1 0:24.86 antsApplyTransf
89899 oesteban 20 0 79592 49952 10648 R 1.6 0.1 0:24.74 antsApplyTransf
89914 oesteban 20 0 79592 50020 10732 R 1.6 0.1 0:24.58 antsApplyTransf
89935 oesteban 20 0 78964 50020 10732 R 1.6 0.1 0:24.42 antsApplyTransf
90163 oesteban 20 0 78964 49824 10732 R 1.6 0.1 0:21.76 antsApplyTransf
90172 oesteban 20 0 78964 50020 10732 R 1.6 0.1 0:21.69 antsApplyTransf
90189 oesteban 20 0 78964 50060 10732 R 1.6 0.1 0:20.08 antsApplyTransf
90207 oesteban 20 0 78964 50020 10732 R 1.6 0.1 0:19.48 antsApplyTransf
90222 oesteban 20 0 78964 49828 10732 R 1.6 0.1 0:19.01 antsApplyTransf
90235 oesteban 20 0 78964 49824 10732 R 1.6 0.1 0:18.64 antsApplyTransf
90257 oesteban 20 0 78964 49936 10732 R 1.6 0.1 0:17.75 antsApplyTransf
90276 oesteban 20 0 78964 49828 10732 R 1.6 0.1 0:16.57 antsApplyTransf
90302 oesteban 20 0 78964 49956 10652 R 1.6 0.1 0:16.36 antsApplyTransf
90322 oesteban 20 0 78964 49956 10652 R 1.6 0.1 0:16.29 antsApplyTransf
90326 oesteban 20 0 78964 50020 10732 R 1.6 0.1 0:16.27 antsApplyTransf
90387 oesteban 20 0 78964 50020 10732 R 1.6 0.1 0:15.92 antsApplyTransf
90434 oesteban 20 0 78964 50024 10732 R 1.6 0.1 0:15.41 antsApplyTransf
90448 oesteban 20 0 78964 49952 10652 R 1.6 0.1 0:14.60 antsApplyTransf
90474 oesteban 20 0 78964 50020 10732 R 1.6 0.1 0:10.42 antsApplyTransf
90542 oesteban 20 0 78964 49728 10732 R 1.6 0.1 0:08.12 antsApplyTransf
90546 oesteban 20 0 78964 49756 10732 R 1.6 0.1 0:08.09 antsApplyTransf
90565 oesteban 20 0 78964 49824 10732 R 1.6 0.1 0:08.01 antsApplyTransf
90569 oesteban 20 0 78964 49756 10732 R 1.6 0.1 0:07.98 antsApplyTransf
90581 oesteban 20 0 78964 49756 10732 R 1.6 0.1 0:07.91 antsApplyTransf
90593 oesteban 20 0 78964 49756 10732 R 1.6 0.1 0:07.73 antsApplyTransf
90636 oesteban 20 0 78964 49560 10732 R 1.6 0.1 0:06.95 antsApplyTransf
90642 oesteban 20 0 78964 49840 10732 R 1.6 0.1 0:06.91 antsApplyTransf
90677 oesteban 20 0 78964 49472 10564 R 1.6 0.1 0:06.46 antsApplyTransf
90686 oesteban 20 0 78964 49500 10732 R 1.6 0.1 0:06.41 antsApplyTransf
90697 oesteban 20 0 78964 49828 10660 R 1.6 0.1 0:06.31 antsApplyTransf
90785 oesteban 20 0 78964 48704 10732 R 1.6 0.0 0:03.28 antsApplyTransf
90822 oesteban 20 0 78964 48436 10676 R 1.6 0.0 0:01.03 antsApplyTransf
90879 oesteban 20 0 45868 14864 8684 R 1.6 0.0 0:00.07 antsApplyTransf
39577 oesteban 20 0 1021504 509948 10356 R 1.2 0.5 1:45.19 fmriprep
39598 oesteban 20 0 761848 249624 9628 R 1.2 0.3 1:05.81 fmriprep
89847 oesteban 20 0 79592 50048 10744 R 1.2 0.1 0:24.98 antsApplyTransf
89885 oesteban 20 0 79592 50088 10780 R 1.2 0.1 0:24.77 antsApplyTransf
89900 oesteban 20 0 79592 50024 10732 R 1.2 0.1 0:24.72 antsApplyTransf
89924 oesteban 20 0 78964 49824 10732 R 1.2 0.1 0:24.52 antsApplyTransf
90144 oesteban 20 0 78964 50020 10732 R 1.2 0.1 0:22.30 antsApplyTransf
90288 oesteban 20 0 78964 50020 10732 R 1.2 0.1 0:16.48 antsApplyTransf
90338 oesteban 20 0 78964 49824 10732 R 1.2 0.1 0:16.15 antsApplyTransf
90351 oesteban 20 0 78964 49824 10732 R 1.2 0.1 0:16.06 antsApplyTransf
90375 oesteban 20 0 78964 49828 10732 R 1.2 0.1 0:15.96 antsApplyTransf
90386 oesteban 20 0 78964 50020 10732 R 1.2 0.1 0:15.93 antsApplyTransf
90401 oesteban 20 0 78964 49980 10732 R 1.2 0.1 0:15.82 antsApplyTransf
90420 oesteban 20 0 78964 50020 10732 R 1.2 0.1 0:15.76 antsApplyTransf