Running fmriprep on M1 chip Mac Mini (using Docker Wrapper)

[JuYuqian]

I was trying to use fmriprep on my M1 chip Mac Mini with Docker Wrapper. After I typed in the command in terminal, it started to test BIDS compatibility. It went well till saying 'Slic-timing correction will be included', then error messages pop up.

I wonder if it's a M1 chip issue or I did something wrong. I actually successfully did the same preprocessing using fmriprep on another Windows 10 Laptop.

Any help is appreciated.

Below is my command and the error messages:

(base) yuqianni@Yuqians-Mac-mini ~ % fmriprep-docker /Users/yuqianni/Desktop/affective_video_raw_data /Users/yuqianni/Desktop/affective_video_preprocessing participant --participant-label 02 --fs-license-file /Users/yuqianni/Freesurfer/freesurfer.txt --fd-spike-threshold 0.5 --dvars-spike-threshold 1.5
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Warning: <8GB of RAM is available within your Docker environment.
Some parts of fMRIPrep may fail to complete.
Continue anyway? [y/N]y
RUNNING: docker run --rm -e DOCKER_VERSION_8395080871=20.10.7 -it -v /Users/yuqianni/Freesurfer/freesurfer.txt:/opt/freesurfer/license.txt:ro -v /Users/yuqianni/Desktop/affective_video_raw_data:/data:ro -v /Users/yuqianni/Desktop/affective_video_preprocessing:/out nipreps/fmriprep:20.2.1 /data /out participant --participant-label 02 --fd-spike-threshold 0.5 --dvars-spike-threshold 1.5
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
/usr/local/miniconda/lib/python3.7/site-packages/bids/layout/validation.py:46: UserWarning: The ability to pass arguments to BIDSLayout that control indexing is likely to be removed in future; possibly as early as PyBIDS 0.14. This includes the `config_filename`, `ignore`, `force_index`, and `index_metadata` arguments. The recommended usage pattern is to initialize a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout via the `indexer` argument.
  warnings.warn("The ability to pass arguments to BIDSLayout that control "
bids-validator@1.4.0


This dataset appears to be BIDS compatible.

        Summary:                Available Tasks:            Available Modalities: 
        56 Files, 2.77GB        Functional localizer        T1w                   
        11 - Subjects           Passive Viewing             bold                  
        1 - Session                                                               


	If you have any questions, please post on https://neurostars.org/tags/bids.

210627-02:47:56,244 nipype.workflow IMPORTANT:
	 
    Running fMRIPREP version 20.2.1:
      * BIDS dataset path: /data.
      * Participant list: ['02'].
      * Run identifier: 20210627-024727_75af46ec-dd70-4569-b17a-3f5aa343f084.
      * Output spaces: MNI152NLin2009cAsym:res-native.
      * Pre-run FreeSurfer's SUBJECTS_DIR: /out/freesurfer.
210627-02:47:58,315 nipype.workflow INFO:
	 No single-band-reference found for sub-02_task-functionallocalizer_run-01_bold.nii.gz.
210627-02:47:59,929 nipype.workflow IMPORTANT:
	 Slice-timing correction will be included.
210627-02:48:00,252 nipype.workflow INFO:
	 No single-band-reference found for sub-02_task-view_run-01_bold.nii.gz.
210627-02:48:01,632 nipype.workflow IMPORTANT:
	 Slice-timing correction will be included.
210627-02:48:01,953 nipype.workflow INFO:
	 No single-band-reference found for sub-02_task-view_run-02_bold.nii.gz.
210627-02:48:03,337 nipype.workflow IMPORTANT:
	 Slice-timing correction will be included.
Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/workflow.py", line 82, in build_workflow
    retval["workflow"] = init_fmriprep_wf()
  File "<string>", line 2, in __setitem__
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/managers.py", line 796, in _callmethod
    kind, result = conn.recv()
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/connection.py", line 383, in _recv
    raise EOFError
EOFError
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/managers.py", line 788, in _callmethod
    conn = self._tls.connection
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/miniconda/bin/fmriprep", line 10, in <module>
    sys.exit(main())
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py", line 45, in main
    fmriprep_wf = retval.get("workflow", None)
  File "<string>", line 2, in get
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/managers.py", line 792, in _callmethod
    self._connect()
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/managers.py", line 779, in _connect
    conn = self._Client(self._token.address, authkey=self._authkey)
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/connection.py", line 492, in Client
    c = SocketClient(address)
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/connection.py", line 619, in SocketClient
    s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
Sentry is attempting to send 0 pending error messages
Waiting up to 2 seconds
Press Ctrl-C to quit
fMRIPrep: Please report errors to https://github.com/nipreps/fmriprep/issues

[effigies]

I haven't seen anything like this before. It appears to be synchronization issue, but it's internal to the Python multiprocessing API, so I'm not sure how we would fix it.

Does this happen reliably?

[JuYuqian]

Yes, I tried multiple times. and changed the subject labels, the issue still exists.

[effigies]

I'm going to (reluctantly) ping @oesteban. There's something nasty going on with process managers, but Googling and a little bit of digging into CPython is not buying me much clarity.

[oesteban]

Perhaps overriding the use of forkserver?

[mgxd]

I wonder if we even need to set the multiprocessing start method anymore, now that we can read/write to the config file.

[oesteban]

I wonder if we even need to set the multiprocessing start method anymore, now that we can read/write to the config file.

Yes, we still need to do this.

The reason being that we want to start the forkserver as soon as possible in the runtime, when the memory fingerprint is minimal (typically ~850MB at that point on a Linux box). We want to use the forkserver because it starts as many workers as you request, and when the workers are cleansed (and replaced with a fresh one), they start with the same memory fingerprint of the original fork (remember, ~850MB for fMRIPrep when the forkserver is initiated).

On Mac this should not be necessary in principle. 1) I think mac uses spawn to start subprocesses, so there's no step where virtual memory blows up when forking; 2) even if it used fork, the problem comes from how CPython handles memory in Linux (it does not release memory from previously used imports, which is not the case in Windows and, probably -I'm not sure- in mac).

Long story short, forkserver is still necessary on Linux.

[effigies]

The problem is that they're using docker on Mac, so we can't dispense with forkserver. The underlying Python is still Linux.

[oesteban]

Ugh - thought it was baremetal installation.

[JuYuqian]

Sorry about confusion. I'm using the Docker Wrapper.

[effigies]

@JuYuqian, I think I have a fix in #2435. Would you give the following a try?

git clone https://github.com/nipreps/fmriprep.git
git -C fmriprep fetch origin pull/2435/head
git -C fmriprep checkout FETCH_HEAD

You should see something like:

Cloning into 'fmriprep'...
remote: Enumerating objects: 30475, done.
remote: Counting objects: 100% (242/242), done.
remote: Compressing objects: 100% (115/115), done.
remote: Total 30475 (delta 143), reused 206 (delta 127), pack-reused 30233
Receiving objects: 100% (30475/30475), 40.23 MiB | 14.21 MiB/s, done.
Resolving deltas: 100% (22401/22401), done.
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (1/1), done.
remote: Total 5 (delta 4), reused 5 (delta 4), pack-reused 0
Unpacking objects: 100% (5/5), 717 bytes | 717.00 KiB/s, done.
From https://github.com/nipreps/fmriprep
 * branch              refs/pull/2435/head -> FETCH_HEAD
Note: switching to 'FETCH_HEAD'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 314dc39a FIX: Fall-back to initializing workflow in main process

Then you can patch this into the Docker container with fmriprep-docker as follows:

fmriprep-docker --patch fmriprep=fmriprep/fmriprep [...]

Where [...] is the rest of your usual command.

[JuYuqian]

Thank you, @effigies ,

After 'cloning into 'fmriprep......'', I tried these two commands, but neither works. Maybe I typed the command in a wrong way?

fmriprep-docker --patch fmriprep=fmriprep/fmriprep /Users/yuqianni/Desktop/affective_video_raw_data /Users/yuqianni/Desktop/affective_video_preprocessing participant --participant-label 01 --fs-license-file /Users/yuqianni/Freesurfer/freesurfer.txt

fmriprep-docker --patch fmriprep=fmriprep/fmriprep fmriprep-docker /Users/yuqianni/Desktop/affective_video_raw_data /Users/yuqianni/Desktop/affective_video_preprocessing participant --participant-label 01 --fs-license-file /Users/yuqianni/Freesurfer/freesurfer.txt

Both command gave me this error:

Traceback (most recent call last):
  File "/Users/yuqianni/.local/bin/fmriprep-docker", line 8, in <module>
    sys.exit(main())
  File "/Users/yuqianni/.local/lib/python3.8/site-packages/fmriprep_docker.py", line 349, in main
    opts, unknown_args = parser.parse_known_args()
  File "/Users/yuqianni/opt/anaconda3/lib/python3.8/argparse.py", line 1800, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/Users/yuqianni/opt/anaconda3/lib/python3.8/argparse.py", line 2006, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/Users/yuqianni/opt/anaconda3/lib/python3.8/argparse.py", line 1946, in consume_optional
    take_action(action, args, option_string)
  File "/Users/yuqianni/opt/anaconda3/lib/python3.8/argparse.py", line 1874, in take_action
    action(self, namespace, argument_values, option_string)
  File "/Users/yuqianni/.local/lib/python3.8/site-packages/fmriprep_docker.py", line 236, in __call__
    k, v = kv.split("=")
ValueError: not enough values to unpack (expected 2, got 1)

[effigies]

Oh, you may need to put the patch flag at the end, not the beginning.