fsl.MotionOutliers fails to run when inputs.threshold is set

[beausievers]

fsl.MotionOutliers throws a FileNotFoundError regarding a file in a temporary directory when inputs.threshold is set to a specific value. When inputs.threshold is not set, it runs just fine. This behavior persists for different values of inputs.metric.

Simple example:

from nipype.interfaces import fsl
from nipype import Node
import os

# Works

epi_path = 'my_epi.nii'
motion_assess = Node(fsl.MotionOutliers(), name='motion_assess')
motion_assess.inputs.in_file = os.path.abspath(epi_path)
motion_assess.inputs.metric = 'fd'
#motion_assess.inputs.threshold = 0.9
motion_assess.run()

# Doesn't work

epi_path = 'my_epi.nii'
motion_assess = Node(fsl.MotionOutliers(), name='motion_assess')
motion_assess.inputs.in_file = os.path.abspath(epi_path)
motion_assess.inputs.metric = 'fd'
motion_assess.inputs.threshold = 0.9
motion_assess.run()

Here is the traceback:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-56-1bab1371cb85> in <module>()
      6 motion_assess.inputs.metric = 'fd'
      7 motion_assess.inputs.threshold = 0.9
----> 8 motion_assess.run()

/usr/local/lib/python2.7/site-packages/nipype/pipeline/engine.pyc in run(self, updatehash)
   1426                     self.inputs.get_traitsfree())
   1427             try:
-> 1428                 self._run_interface()
   1429             except:
   1430                 os.remove(hashfile_unfinished)

/usr/local/lib/python2.7/site-packages/nipype/pipeline/engine.pyc in _run_interface(self, execute, updatehash)
   1536         old_cwd = os.getcwd()
   1537         os.chdir(self.output_dir())
-> 1538         self._result = self._run_command(execute)
   1539         os.chdir(old_cwd)
   1540

/usr/local/lib/python2.7/site-packages/nipype/pipeline/engine.pyc in _run_command(self, execute, copyfiles)
   1662                 logger.info('Running: %s' % cmd)
   1663             try:
-> 1664                 result = self._interface.run()
   1665             except Exception, msg:
   1666                 self._result.runtime.stderr = msg

/usr/local/lib/python2.7/site-packages/nipype/interfaces/base.pyc in run(self, **inputs)
   1032         try:
   1033             runtime = self._run_wrapper(runtime)
-> 1034             outputs = self.aggregate_outputs(runtime)
   1035             runtime.endTime = dt.isoformat(dt.utcnow())
   1036             timediff = parseutc(runtime.endTime) - parseutc(runtime.startTime)

/usr/local/lib/python2.7/site-packages/nipype/interfaces/base.pyc in aggregate_outputs(self, runtime, needed_outputs)
   1126                         msg = ("File/Directory '%s' not found for %s output "
   1127                                "'%s'." % (val, self.__class__.__name__, key))
-> 1128                         raise FileNotFoundError(msg)
   1129                     else:
   1130                         raise error

FileNotFoundError: File/Directory '/private/var/folders/q4/_36dqfr17d92b3hxw01gbphm0000gn/T/tmpySfAEO/motion_assess/my_epi_outliers.txt' not found for MotionOutliers output 'out_file'.
Interface MotionOutliers failed to run.

I'm running Mac OS 10.9.5, with FSL version 5.0.9.

nipype.get_info() output:

{'commit_hash': '7c7b0a7',
 'commit_source': 'repository',
 'networkx_version': '1.11',
 'nibabel_version': '2.0.2',
 'numpy_version': '1.11.0',
 'pkg_path': '/usr/local/lib/python2.7/site-packages/nipype',
 'scipy_version': '0.17.0',
 'sys_executable': '/usr/local/opt/python/bin/python2.7',
 'sys_platform': 'darwin',
 'sys_version': '2.7.11 (default, Dec  5 2015, 22:44:43) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]',
 'traits_version': '4.5.0'}

[beausievers]

This may be because fsl_motion_outliers is not correctly creating the text file containing the motion confounds. With FSL 5.0.9, the following command correctly creates the motion_confound.txt file:

fsl_motion_outliers -i my_epi.nii -o motion_confound.txt -p fd_plot.png --fd

Whereas the following command does not:

fsl_motion_outliers -i my_epi.nii -o motion_confound.txt -p fd_plot.png --fd --thresh=0.9

So perhaps this is a bug in FSL 5.0.9, an issue specific to my NIfTI image, or something wrong with my installation.

[satra]

@beausievers - both commands work for me on a linux centos 7.2 platform with fsl 5.0.9. perhaps its something specific to your my_epi.nii file or your fsl installation?

$ fsl_motion_outliers -i bold.nii.gz -o motion_confound.txt -p fd_plot.png --fd --thresh=0.9
$ ls -alrt
-rw-r--r--  1 satra om_scratch     2327 Apr 10 08:10 fd_plot.png
-rw-r--r--  1 satra om_scratch     1872 Apr 10 08:11 motion_confound.txt

[beausievers]

@satra After doing some more testing and getting help on the FSL list, it looks like fsl_motion_outliers is not guaranteed to create an output text file. If the NIfTI image contains no timepoints above the threshold, fsl_motion_outliers will terminate with no output.

Creates output:

fsl_motion_outliers -i my_epi.nii -o motion_confound.txt --fd --thresh=0.01

Does not create output:

fsl_motion_outliers -i my_epi.nii -o motion_confound.txt --fd --thresh=1.0

I am not super familiar with the NiPype internals, so apologies if this comment belies my novice status, but my intuition is that a NiPype Node using fsl.MotionOutliers should take this possibility into account instead of throwing a FileNotFoundError when no timepoints are above the threshold.

[satra]

@beausievers - thanks for the feedback. it should be taken care of within nipype.

as a first pass can you change this line:

https://github.com/nipy/nipype/blob/master/nipype/interfaces/fsl/utils.py#L2048

to:

out_file = File()

first to see if this helps.

if so, would you be willing to submit a pull-request for this change? we can guide you.

[beausievers]

@satra This appears to fix the FileNotFoundError problem. I've created a pull request here:

#1428

Thanks, and please let me know if I didn't do that right.

[satra]

@beausievers - thank you.

one more check. if running as interface:

res = interface.run()

res.outputs.out_file should be Undefined

if running in a workflow:

check the report.rst file to see if the output_file output is undefined.

[beausievers]

@satra res.outputs.out_file is not Undefined, but instead is %s_outliers.txt. Looking at the MotionOutliers classes, there is no obvious change to make in the input or output spec that would change the out_file to Undefined if fsl_motion_outliers produces no output. I fear I do not understand NiPype well enough to fix this myself...

[satra]

@beausievers - thanks for taking a look - yes - this will require a bit of work on our part. let me look into this tomorrow.

now one issue is that in the context of a workflow having that output be sometimes defined and sometimes not may be an issue, depending on what it is connected to.