spm.DARTEL() ; spm.DARTELNorm2MNI() - unable to use files located in folders different than current working directory

[gacek91]

Summary

While running spm.DARTEL(), I'm getting an unexpected error when Template_6.nii is created. No error occurs with Templates 0-5.

Edit: similar error occurs when I try to use a different function: spm.DARTELNorm2MNI()

Actual behavior

Error output below:

---------------------------------------------------------------------------
TraitError                                Traceback (most recent call last)
<ipython-input-28-3d03449a9e1d> in <module>()
      1 dartel = spm.DARTEL()
      2 dartel.inputs.image_files = [rc1DartelPaths,rc2DartelPaths]
----> 3 dartel.run()

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/nipype/interfaces/base/core.py in run(self, cwd, ignore_exception, **inputs)
    519         try:
    520             runtime = self._run_interface(runtime)
--> 521             outputs = self.aggregate_outputs(runtime)
    522         except Exception as e:
    523             import traceback

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/nipype/interfaces/base/core.py in aggregate_outputs(self, runtime, needed_outputs)
    616                                "'%s'." % (val, self.__class__.__name__, key))
    617                         raise FileNotFoundError(msg)
--> 618                     raise error
    619 
    620         return outputs

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/nipype/interfaces/base/core.py in aggregate_outputs(self, runtime, needed_outputs)
    609                                                       self.__class__.__name__))
    610                 try:
--> 611                     setattr(outputs, key, val)
    612                 except TraitError as error:
    613                     if getattr(error, 'info',

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/nipype/interfaces/base/traits_extension.py in validate(self, object, name, value)
    110                 args='The trait \'{}\' of {} instance is {}, but the path '
    111                 ' \'{}\' does not exist.'.format(name, class_of(object),
--> 112                                                  self.info_text, value))
    113 
    114         self.error(object, name, value)

TraitError: The trait 'final_template_file' of a DARTELOutputSpec instance is an existing file name, but the path  '/Users/Gacek/Documents/VBM/Template_6.nii' does not exist.

Expected behavior

Obviously no error. I can see that TraitError refers to a non-existent file which is supposed to be in my current working directory (when in fact all the templates are saved in ../segmentation/dartel/, which is the place where my rc1 and rc2 files are stored).

How to replicate the behavior

Put rc1 and rc2 files in a folder (change pth and niiFnames variables accordingly)

Script/Workflow details

import nipype.interfaces.spm as spm
import os, os.path as op, shutil
import nipype
from nipype.interfaces.matlab import MatlabCommand


pth = op.join(os.getcwd(),'GL','niftis_py')

niiFnames = [f for f in os.listdir(pth) if f.endswith('.nii') if f.startswith('GL')]
niiPaths = [op.join(pth,nii) for nii in niiFnames]
segmNativeFnames = [c + nii for c in ['c1','c2','c3'] for nii in niiFnames]
segmDartelFnames = [rc + nii for rc in ['rc1','rc2'] for nii in niiFnames]

segmNativePathsNew = [op.join(pth,'segmentation','native', c) for c in segmNativeFnames]
segmDartelPathsNew = [op.join(pth,'segmentation','dartel',rc) for rc in segmDartelFnames]
rc1DartelPaths = [rc for rc in segmDartelPathsNew if op.split(rc)[-1].startswith('rc1')]
rc2DartelPaths = [rc for rc in segmDartelPathsNew if op.split(rc)[-1].startswith('rc2')]

dartel = spm.DARTEL()
dartel.inputs.image_files = [rc1DartelPaths,rc2DartelPaths]
dartel.run()

Platform details:

'pkg_path': '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/nipype', 'commit_source': 'installation', 'commit_hash': '7d3966f', 'nipype_version': '1.0.3', 'sys_version': '3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 03:03:55) \n[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]', 'sys_executable': '/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6', 'sys_platform': 'darwin', 'numpy_version': '1.13.3', 'scipy_version': '0.19.0', 'networkx_version': '2.1', 'nibabel_version': '2.2.0', 'traits_version': '4.6.0'

Execution environment

Choose one

• Container [Tag: ???]
• My python environment inside container [Base Tag: ???]
• My python environment outside container

[mgxd]

hi @gacek91 - thanks for the report, you are right that the interface is not flexible enough to handle rc files located in different directories. I'm not very familiar with dartel - are the template files always generated in the directory where the rc files are located? What if rc1 and rc2 are not in the same directory?

for a quick workaround, could you try running the interface within a Node?

from nipype import Node
dartel = Node(spm.DARTEL(), name='dartel')
dartel.inputs.image_files = [rc1DartelPaths,rc2DartelPaths]
res = dartel.run()

# template files
res.outputs.template_files
# final template
res.outputs.final_template_file

[gacek91]

Hey @mgxd,

Yeah, template files are generated in the very same place where rc files are.

Node doesn't work either, but let's call it progress anyway because it gave me a new error:

180518-08:15:48,275 workflow INFO:
	 [Node] Setting-up "dartel" in "/var/folders/1b/t25k6mnn6gs8k_w3yf0g3bv40000gn/T/tmp0tp_9he7/dartel".
180518-08:15:48,292 workflow INFO:
	 [Node] Running "dartel" ("nipype.interfaces.spm.preprocess.DARTEL")
180518-08:31:01,569 workflow WARNING:
	 [Node] Error on "dartel" (/var/folders/1b/t25k6mnn6gs8k_w3yf0g3bv40000gn/T/tmp0tp_9he7/dartel)
---------------------------------------------------------------------------
TraitError                                Traceback (most recent call last)
<ipython-input-17-fcdb9ab76685> in <module>()
      2 dartel = Node(spm.DARTEL(), name='dartel')
      3 dartel.inputs.image_files = [rc1DartelPaths,rc2DartelPaths]
----> 4 res = dartel.run()
      5 
      6 # template files

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py in run(self, updatehash)
    478 
    479         try:
--> 480             result = self._run_interface(execute=True)
    481         except Exception:
    482             logger.warning('[Node] Error on "%s" (%s)', self.fullname, outdir)

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py in _run_interface(self, execute, updatehash)
    562             self._update_hash()
    563             return self._load_results()
--> 564         return self._run_command(execute)
    565 
    566     def _load_results(self):

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py in _run_command(self, execute, copyfiles)
    660             self.config,
    661             dirs2keep=dirs2keep)
--> 662         _save_resultfile(result, outdir, self.name)
    663 
    664         return result

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/nipype/pipeline/engine/utils.py in save_resultfile(result, cwd, name)
    242         except AttributeError:
    243             outputs = result.outputs.dictcopy()  # outputs was a bunch
--> 244         result.outputs.set(**modify_paths(outputs, relative=True, basedir=cwd))
    245 
    246     savepkl(resultsfile, result)

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/traits/util/deprecated.py in wrapper(*args, **kw)
     30 
     31             warnings.warn(message, DeprecationWarning, stacklevel=2)
---> 32             return fn(*args, **kw)
     33 
     34         return wrapper

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/traits/has_traits.py in set(self, trait_change_notify, **traits)
   1549     def set ( self, trait_change_notify = True, **traits ):
   1550         return self.trait_set(
-> 1551             trait_change_notify=trait_change_notify, **traits)
   1552 
   1553     set.__doc__ = trait_set.__doc__

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/traits/has_traits.py in trait_set(self, trait_change_notify, **traits)
   1541         else:
   1542             for name, value in list(traits.items()):
-> 1543                 setattr( self, name, value )
   1544 
   1545         return self

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/traits/trait_types.py in validate(self, object, name, value)
   2334                 return value
   2335 
-> 2336             return TraitListObject( self, object, name, value )
   2337 
   2338         self.error( object, name, value )

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/traits/trait_handlers.py in __init__(self, trait, object, name, value)
   2311             except TraitError as excp:
   2312                 excp.set_prefix( 'Each element of the' )
-> 2313                 raise excp
   2314 
   2315         self.len_error( len( value ) )

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/traits/trait_handlers.py in __init__(self, trait, object, name, value)
   2303                 validate = trait.item_trait.handler.validate
   2304                 if validate is not None:
-> 2305                     value = [ validate( object, name, val ) for val in value ]
   2306 
   2307                 list.__setitem__(self, slice(0, 0), value )

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/traits/trait_handlers.py in <listcomp>(.0)
   2303                 validate = trait.item_trait.handler.validate
   2304                 if validate is not None:
-> 2305                     value = [ validate( object, name, val ) for val in value ]
   2306 
   2307                 list.__setitem__(self, slice(0, 0), value )

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/nipype/interfaces/base/traits_extension.py in validate(self, object, name, value)
    110                 args='The trait \'{}\' of {} instance is {}, but the path '
    111                 ' \'{}\' does not exist.'.format(name, class_of(object),
--> 112                                                  self.info_text, value))
    113 
    114         self.error(object, name, value)

TraitError: The trait 'dartel_flow_fields' of a DARTELOutputSpec instance is an existing file name, but the path  '/private/var/folders/1b/t25k6mnn6gs8k_w3yf0g3bv40000gn/T/tmp0tp_9he7/dartel/u_rc1Glab_3004_1_Template.nii' does not exist.

Back to the original code: Not sure where to look for this bug in Dartel's code, but it's pretty clear that it looks for a template file in os.getcwd(), which in my case is not the same place where my rc and template files are to be found.

The simple workaround is just running os.chdir() first to change my directory to where the files are located, but I wouldn't mind not having to do that.

To sum it up: I managed to get it to work only when I changed the directory, but the code should be looking for something like:
op.join(op.split('/place/where/the/files/are/rc1.nii')[0],'Template_6.nii') and not: op.join(os.getcwd(),'Template_6.nii')

[gacek91]

Hey there, heads up: same issue occurs with spm.DARTELNorm2MNI().

Regards

[satra]

there are two workflows which used to work fine:

https://github.com/nipy/nipype/blob/master/nipype/workflows/fmri/spm/preprocess.py#L131
https://github.com/nipy/nipype/blob/master/nipype/workflows/fmri/spm/preprocess.py#L234

that perhaps can both be tested and looked at as to why these interfaces are behaving differently right now.