ICA_AROMA() not working

[debottamKundu]

Summary

The newly released version 0.13.0 has a ICA_AROMA() interface defined in the fsl module but although the AROMA is executing correctly, it is throwing an error when passing the output of a node that contains the interface of the AROMA module to another node.

Actual behavior

AROMA node is throwing an "NameError: global name 'outputs' is not defined
Interface ICA_AROMA failed to run" although all the files that are supposed to be generated by AROMA are already in the output directory which is specified via the input.

170516-20:56:40,957 workflow INFO:
Executing node a in dir: /tmp/tmppbBpxu/a
170516-20:56:40,974 workflow INFO:
Running: ICA_AROMA.py -den nonaggr -feat /home/debottam/Desktop/OUTPUT.feat -o /home/debottam/Desktop/OUTPUT.feat/ICA_testout/
170516-21:08:56,984 interface INFO:
stdout 2017-05-16T21:08:56.983846:
170516-21:08:56,986 interface INFO:
stdout 2017-05-16T21:08:56.983846:------------------------------- RUNNING ICA-AROMA -------------------------------
170516-21:08:56,989 interface INFO:
stdout 2017-05-16T21:08:56.983846:--------------- 'ICA-based Automatic Removal Of Motion Artifacts' ---------------
170516-21:08:56,992 interface INFO:
stdout 2017-05-16T21:08:56.983846:
170516-21:08:57,1 interface INFO:
stdout 2017-05-16T21:08:56.983846:Step 1) MELODIC
170516-21:08:57,3 interface INFO:
stdout 2017-05-16T21:08:56.983846: - The existing/specified MELODIC directory will be used.
170516-21:08:57,5 interface INFO:
stdout 2017-05-16T21:08:56.983846: - The MELODIC directory does not contain the required 'stats' folder. Mixture modeling on the Z-statistical maps will be run.
170516-21:08:57,6 interface INFO:
stdout 2017-05-16T21:08:56.983846:Step 2) Automatic classification of the components
170516-21:08:57,8 interface INFO:
stdout 2017-05-16T21:08:56.983846: - registering the spatial maps to MNI
170516-21:08:57,10 interface INFO:
stdout 2017-05-16T21:08:56.983846: - extracting the CSF & Edge fraction features
170516-21:08:57,14 interface INFO:
stdout 2017-05-16T21:08:56.983846: - extracting the Maximum RP correlation feature
170516-21:08:57,17 interface INFO:
stdout 2017-05-16T21:08:56.983846: - extracting the High-frequency content feature
170516-21:08:57,18 interface INFO:
stdout 2017-05-16T21:08:56.983846: - classification
170516-21:08:57,21 interface INFO:
stdout 2017-05-16T21:08:56.983846:Step 3) Data denoising
170516-21:08:57,25 interface INFO:
stdout 2017-05-16T21:08:56.983846:
170516-21:08:57,28 interface INFO:
stdout 2017-05-16T21:08:56.983846:----------------------------------- Finished -----------------------------------
170516-21:08:57,30 interface INFO:
stdout 2017-05-16T21:08:56.983846:

NameError Traceback (most recent call last)
in ()
----> 1 a.run()

/home/debottam/anaconda2/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.pyc in run(self, updatehash)
370 self.inputs.get_traitsfree())
371 try:
--> 372 self._run_interface()
373 except:
374 os.remove(hashfile_unfinished)

/home/debottam/anaconda2/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.pyc in _run_interface(self, execute, updatehash)
480 old_cwd = os.getcwd()
481 os.chdir(self.output_dir())
--> 482 self._result = self._run_command(execute)
483 os.chdir(old_cwd)
484

/home/debottam/anaconda2/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.pyc in _run_command(self, execute, copyfiles)
611 logger.info('Running: %s' % cmd)
612 try:
--> 613 result = self._interface.run()
614 except Exception as msg:
615 self._save_results(result, cwd)

/home/debottam/anaconda2/lib/python2.7/site-packages/nipype/interfaces/base.pyc in run(self, **inputs)
1080 try:
1081 runtime = self._run_wrapper(runtime)
-> 1082 outputs = self.aggregate_outputs(runtime)
1083 runtime.endTime = dt.isoformat(dt.utcnow())
1084 timediff = parseutc(runtime.endTime) - parseutc(runtime.startTime)

/home/debottam/anaconda2/lib/python2.7/site-packages/nipype/interfaces/base.pyc in aggregate_outputs(self, runtime, needed_outputs)
1151 """ Collate expected outputs and check for existence
1152 """
-> 1153 predicted_outputs = self._list_outputs()
1154 outputs = self._outputs()
1155 if predicted_outputs:

/home/debottam/anaconda2/lib/python2.7/site-packages/nipype/interfaces/fsl/ICA_AROMA.pyc in _list_outputs(self)
109 def _list_outputs(self):
110 out_dir = os.path.abspath(self.inputs.out_dir)
--> 111 outputs['out_dir'] = out_dir
112
113 if self.inputs.denoise_type in ('aggr', 'both'):

NameError: global name 'outputs' is not defined
Interface ICA_AROMA failed to run.

Expected behavior

The outputs is supposed to return the output directory containing the AROMA output

How to replicate the behavior

FEAT is already ran on requisite images and a FEAT folder containing the FEAT files is passed to the AROMA module. An already existing output directory is also passed.

Script/Workflow details

from nipype.pipeline.engine import Node
from nipype import Function
from nipype.interfaces.fsl.maths import TemporalFilter
from nipype.interfaces.fsl import ICA_AROMA
a = Node(ICA_AROMA.ICA_AROMA(),name='a')
a.inputs.feat_dir = '/home/debottam/Desktop/OUTPUT.feat'
a.inputs.out_dir = '/home/debottam/Desktop/OUTPUT.feat/ICA_testout/'
a.run()

Platform details:

Please paste the output of: python -c "import nipype; print(nipype.get_info()); print(nipype.__version__)"
{'nibabel_version': '2.1.0', 'sys_executable': '/home/debottam/anaconda2/bin/python', 'networkx_version': '1.11', 'numpy_version': '1.11.0', 'sys_platform': 'linux2', 'sys_version': '2.7.13 | packaged by conda-forge | (default, May 2 2017, 12:48:11) \n[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]', 'commit_source': u'installation', 'commit_hash': u'fd5b81b7c', 'pkg_path': '/home/debottam/anaconda2/lib/python2.7/site-packages/nipype', 'nipype_version': u'0.13.0', 'traits_version': '4.6.0', 'scipy_version': '0.19.0'}

Execution environment

Jupyter-Notebook
Choose one

• Container [Tag: Bug]

[chrisgorgo]

@jdkent could you have a look?

I think you just need to add outputs = self.output_spec().get()
here

nipype/nipype/interfaces/fsl/ICA_AROMA.py

Line 111 in 3440181

outputs['out_dir'] = out_dir

[debottamKundu]

I wrote this alternate wrapper with that line included as a modification before; but it was still showing an error

# vi: set ft=python sts=4 ts=4 sw=4 et:
"""This commandline module provides classes for interfacing with the
`ICA-AROMA.py<https://github.com/rhr-pruim/ICA-AROMA>`_ command line tool.
    Change directory to provide relative paths for doctests
    >>> import os
    >>> filepath = os.path.dirname(os.path.realpath(__file__))
    >>> datadir = os.path.realpath(os.path.join(filepath,
    ...                            '../../testing/data'))
    >>> os.chdir(datadir)
"""
from nipype.interfaces.base import (
    TraitedSpec,
    CommandLineInputSpec,
    CommandLine,
    File,
    Directory,
    traits,
    OutputMultiPath
)
import os

class ICA_AROMAaltInputSpec(CommandLineInputSpec):
    feat_dir = Directory(exists=True, mandatory=True,
                         argstr='-feat %s',
                         xor=['in_file', 'mat_file', 'fnirt_warp_file', 'motion_parameters'],
                         desc='If a feat directory exists and temporal filtering '
                         'has not been run yet, ICA_AROMAalt can use the files in '
                         'this directory.')
    in_file = File(exists=True, mandatory=True,
                   argstr='-i %s', xor=['feat_dir'],
                   desc='volume to be denoised')
    out_dir = Directory('out', mandatory=True,
                        argstr='-o %s',
                        desc='output directory')
    mask = File(exists=True, argstr='-m %s', xor=['feat_dir'],
                desc='path/name volume mask')
    dim = traits.Int(argstr='-dim %d',
                     desc='Dimensionality reduction when running '
                     'MELODIC (defualt is automatic estimation)')
    TR = traits.Float(argstr='-tr %.3f',
                      desc='TR in seconds. If this is not specified '
                      'the TR will be extracted from the '
                      'header of the fMRI nifti file.')
    melodic_dir = Directory(exists=True, argstr='-meldir %s',
                            desc='path to MELODIC directory if MELODIC has already been run')
    mat_file = File(exists=True, argstr='-affmat %s', xor=['feat_dir'],
                    desc='path/name of the mat-file describing the '
                    'affine registration (e.g. FSL FLIRT) of the '
                    'functional data to structural space (.mat file)')
    fnirt_warp_file = File(exists=True, argstr='-warp %s', xor=['feat_dir'],
                           desc='File name of the warp-file describing '
                           'the non-linear registration (e.g. FSL FNIRT) '
                           'of the structural data to MNI152 space (.nii.gz)')
    motion_parameters = File(exists=True, mandatory=True,
                             argstr='-mc %s', xor=['feat_dir'],
                             desc='motion parameters file')
    denoise_type = traits.Enum('nonaggr', 'aggr', 'both', 'no', usedefault=True,
                               mandatory=True, argstr='-den %s',
                               desc='Type of denoising strategy:\n'
                               '-none: only classification, no denoising\n'
                               '-nonaggr (default): non-aggresssive denoising, i.e. partial component regression\n'
                               '-aggr: aggressive denoising, i.e. full component regression\n'
                               '-both: both aggressive and non-aggressive denoising (two outputs)')

class ICA_AROMAaltOutputSpec(TraitedSpec):
    aggr_denoised_file = File(exists=True,
                              desc='if generated: aggressively denoised volume')
    nonaggr_denoised_file = File(exists=True,
                                 desc='if generated: non aggressively denoised volume' )
    out_dir = Directory(exists=True,
                        desc='directory contains (in addition to the denoised files): '
                        'melodic.ica + classified_motion_components + '
                        'classification_overview + feature_scores + melodic_ic_mni)')

class ICA_AROMAalt(CommandLine):
    """
    Interface for the ICA_AROMAalt.py script.
    ICA-AROMA (i.e. 'ICA-based Automatic Removal Of Motion Artifacts') concerns
    a data-driven method to identify and remove motion-related independent
    components from fMRI data. To that end it exploits a small, but robust
    set of theoretically motivated features, preventing the need for classifier
    re-training and therefore providing direct and easy applicability.
    See link for further documentation: https://github.com/rhr-pruim/ICA-AROMA
    Example
    -------
    >>> from nipype.interfaces.fsl import ICA_AROMAalt
    >>> from nipype.testing import example_data
    >>> AROMA_obj = ICA_AROMAalt.ICA_AROMAalt()
    >>> AROMA_obj.inputs.in_file = 'functional.nii'
    >>> AROMA_obj.inputs.mat_file = 'func_to_struct.mat'
    >>> AROMA_obj.inputs.fnirt_warp_file = 'warpfield.nii'
    >>> AROMA_obj.inputs.motion_parameters = 'fsl_mcflirt_movpar.txt'
    >>> AROMA_obj.inputs.mask = 'mask.nii.gz'
    >>> AROMA_obj.inputs.denoise_type = 'both'
    >>> AROMA_obj.inputs.out_dir = 'ICA_testout'
    >>> AROMA_obj.cmdline # doctest: +ALLOW_UNICODE
    'ICA_AROMAalt.py -den both -warp warpfield.nii -i functional.nii -m mask.nii.gz -affmat func_to_struct.mat -mc fsl_mcflirt_movpar.txt -o ICA_testout'
    """
    _cmd = 'ICA_AROMA.py'
    input_spec = ICA_AROMAaltInputSpec
    output_spec = ICA_AROMAaltOutputSpec

    def _list_outputs(self):
    	outputs = self.output_spec.get()
        out_dir = os.path.abspath(self.inputs.out_dir)
        outputs['output_file'] = out_dir

        if self.inputs.denoise_type in ('aggr', 'both'):
            outputs['aggr_denoised_file'] = os.path.join(out_dir, 'denoised_func_data_aggr.nii.gz')
        if self.inputs.denoise_type in ('nonaggr', 'both'):
            outputs['nonaggr_denoised_file'] = os.path.join(out_dir, 'denoised_func_data_nonaggr.nii.gz')

        return `outputs`

But again, after completing the AROMA execution,

170517-01:53:09,47 workflow INFO:
Executing node a in dir: /tmp/tmp2t71Y0/a
170517-01:53:09,141 workflow INFO:
Running: ICA_AROMA.py -den nonaggr -feat /home/debottam/Desktop/OUTPUT.feat -o /home/debottam/Desktop/OUTPUT.feat/ICA_testout/
170517-02:06:37,215 interface INFO:
stdout 2017-05-17T02:06:37.214558:
170517-02:06:37,219 interface INFO:
stdout 2017-05-17T02:06:37.214558:------------------------------- RUNNING ICA-AROMA -------------------------------
170517-02:06:37,221 interface INFO:
stdout 2017-05-17T02:06:37.214558:--------------- 'ICA-based Automatic Removal Of Motion Artifacts' ---------------
170517-02:06:37,222 interface INFO:
stdout 2017-05-17T02:06:37.214558:
170517-02:06:37,224 interface INFO:
stdout 2017-05-17T02:06:37.214558:Step 1) MELODIC
170517-02:06:37,225 interface INFO:
stdout 2017-05-17T02:06:37.214558: - The existing/specified MELODIC directory will be used.
170517-02:06:37,226 interface INFO:
stdout 2017-05-17T02:06:37.214558: - The MELODIC directory does not contain the required 'stats' folder. Mixture modeling on the Z-statistical maps will be run.
170517-02:06:37,227 interface INFO:
stdout 2017-05-17T02:06:37.214558:Step 2) Automatic classification of the components
170517-02:06:37,229 interface INFO:
stdout 2017-05-17T02:06:37.214558: - registering the spatial maps to MNI
170517-02:06:37,235 interface INFO:
stdout 2017-05-17T02:06:37.214558: - extracting the CSF & Edge fraction features
170517-02:06:37,241 interface INFO:
stdout 2017-05-17T02:06:37.214558: - extracting the Maximum RP correlation feature
170517-02:06:37,245 interface INFO:
stdout 2017-05-17T02:06:37.214558: - extracting the High-frequency content feature
170517-02:06:37,247 interface INFO:
stdout 2017-05-17T02:06:37.214558: - classification
170517-02:06:37,250 interface INFO:
stdout 2017-05-17T02:06:37.214558:Step 3) Data denoising
170517-02:06:37,252 interface INFO:
stdout 2017-05-17T02:06:37.214558:
170517-02:06:37,254 interface INFO:
stdout 2017-05-17T02:06:37.214558:----------------------------------- Finished -----------------------------------
170517-02:06:37,257 interface INFO:
stdout 2017-05-17T02:06:37.214558:

TypeError Traceback (most recent call last)
in ()
----> 1 a.run()

/home/debottam/anaconda2/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.pyc in run(self, updatehash)
370 self.inputs.get_traitsfree())
371 try:
--> 372 self._run_interface()
373 except:
374 os.remove(hashfile_unfinished)

/home/debottam/anaconda2/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.pyc in _run_interface(self, execute, updatehash)
480 old_cwd = os.getcwd()
481 os.chdir(self.output_dir())
--> 482 self._result = self._run_command(execute)
483 os.chdir(old_cwd)
484

/home/debottam/anaconda2/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.pyc in _run_command(self, execute, copyfiles)
611 logger.info('Running: %s' % cmd)
612 try:
--> 613 result = self._interface.run()
614 except Exception as msg:
615 self._save_results(result, cwd)

/home/debottam/anaconda2/lib/python2.7/site-packages/nipype/interfaces/base.pyc in run(self, **inputs)
1080 try:
1081 runtime = self._run_wrapper(runtime)
-> 1082 outputs = self.aggregate_outputs(runtime)
1083 runtime.endTime = dt.isoformat(dt.utcnow())
1084 timediff = parseutc(runtime.endTime) - parseutc(runtime.startTime)

/home/debottam/anaconda2/lib/python2.7/site-packages/nipype/interfaces/base.pyc in aggregate_outputs(self, runtime, needed_outputs)
1151 """ Collate expected outputs and check for existence
1152 """
-> 1153 predicted_outputs = self._list_outputs()
1154 outputs = self._outputs()
1155 if predicted_outputs:

aromawrapper.py in _list_outputs(self)

TypeError: unbound method get() must be called with ICA_AROMAaltOutputSpec instance as first argument (got nothing instead)
Interface ICA_AROMAalt failed to run.

[effigies]

output_spec().get(), not output_spec.get()

[jdkent]

Sorry I didn't see this sooner, I believe effigies solution is correct. Should I make a pull request reflecting this change? Is there a way to link the pull request to the bug?

[chrisgorgo]

Yes a new PR would be the best option. You can write in the PR description "Closes #<number>" where "<number>" is the number of the issue your PR is fixing (in this case "2026). Thanks!

…

On Tue, May 16, 2017 at 9:33 PM, James Kent ***@***.***> wrote: Sorry I didn't see this sooner, I believe effigies solution is correct. Should I make a pull request reflecting this change? Is there a way to link the pull request to the bug? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2026 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAOkp77clQQnlmg3WES2KrrI41phf6H4ks5r6nixgaJpZM4Nc31H> .

[debottamKundu]

Hi, I made the suggestion suggested by @effigies but now it is showing a new error.

170517-10:24:24,716 workflow INFO:
Executing node a in dir: /tmp/tmpdq5MAz/a
170517-10:24:24,840 workflow INFO:
Running: ICA_AROMA.py -den nonaggr -feat /home/debottam/Desktop/OUTPUT.feat -o /home/debottam/Desktop/OUTPUT.feat/ICA_testout/
170517-10:36:13,2 interface INFO:
stdout 2017-05-17T10:36:13.002433:
170517-10:36:13,4 interface INFO:
stdout 2017-05-17T10:36:13.002433:------------------------------- RUNNING ICA-AROMA -------------------------------
170517-10:36:13,7 interface INFO:
stdout 2017-05-17T10:36:13.002433:--------------- 'ICA-based Automatic Removal Of Motion Artifacts' ---------------
170517-10:36:13,9 interface INFO:
stdout 2017-05-17T10:36:13.002433:
170517-10:36:13,10 interface INFO:
stdout 2017-05-17T10:36:13.002433:Step 1) MELODIC
170517-10:36:13,12 interface INFO:
stdout 2017-05-17T10:36:13.002433: - The existing/specified MELODIC directory will be used.
170517-10:36:13,14 interface INFO:
stdout 2017-05-17T10:36:13.002433: - The MELODIC directory does not contain the required 'stats' folder. Mixture modeling on the Z-statistical maps will be run.
170517-10:36:13,16 interface INFO:
stdout 2017-05-17T10:36:13.002433:Step 2) Automatic classification of the components
170517-10:36:13,17 interface INFO:
stdout 2017-05-17T10:36:13.002433: - registering the spatial maps to MNI
170517-10:36:13,20 interface INFO:
stdout 2017-05-17T10:36:13.002433: - extracting the CSF & Edge fraction features
170517-10:36:13,23 interface INFO:
stdout 2017-05-17T10:36:13.002433: - extracting the Maximum RP correlation feature
170517-10:36:13,24 interface INFO:
stdout 2017-05-17T10:36:13.002433: - extracting the High-frequency content feature
170517-10:36:13,26 interface INFO:
stdout 2017-05-17T10:36:13.002433: - classification
170517-10:36:13,28 interface INFO:
stdout 2017-05-17T10:36:13.002433:Step 3) Data denoising
170517-10:36:13,30 interface INFO:
stdout 2017-05-17T10:36:13.002433:
170517-10:36:13,32 interface INFO:
stdout 2017-05-17T10:36:13.002433:----------------------------------- Finished -----------------------------------
170517-10:36:13,33 interface INFO:
stdout 2017-05-17T10:36:13.002433:

TraitError Traceback (most recent call last)
in ()
----> 1 a.run()

/home/debottam/anaconda2/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.pyc in run(self, updatehash)
370 self.inputs.get_traitsfree())
371 try:
--> 372 self._run_interface()
373 except:
374 os.remove(hashfile_unfinished)

/home/debottam/anaconda2/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.pyc in _run_interface(self, execute, updatehash)
480 old_cwd = os.getcwd()
481 os.chdir(self.output_dir())
--> 482 self._result = self._run_command(execute)
483 os.chdir(old_cwd)
484

/home/debottam/anaconda2/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.pyc in _run_command(self, execute, copyfiles)
611 logger.info('Running: %s' % cmd)
612 try:
--> 613 result = self._interface.run()
614 except Exception as msg:
615 self._save_results(result, cwd)

/home/debottam/anaconda2/lib/python2.7/site-packages/nipype/interfaces/base.pyc in run(self, **inputs)
1080 try:
1081 runtime = self._run_wrapper(runtime)
-> 1082 outputs = self.aggregate_outputs(runtime)
1083 runtime.endTime = dt.isoformat(dt.utcnow())
1084 timediff = parseutc(runtime.endTime) - parseutc(runtime.startTime)

/home/debottam/anaconda2/lib/python2.7/site-packages/nipype/interfaces/base.pyc in aggregate_outputs(self, runtime, needed_outputs)
1176 raise FileNotFoundError(msg)
1177 else:
-> 1178 raise error
1179 return outputs
1180

TraitError: Cannot set the undefined 'output_file' attribute of a 'ICA_AROMAaltOutputSpec' object.
Interface ICA_AROMAalt failed to run.

I think that the outputs['output_file'] = out_dir needs to be changed to outputs[out_dir]=dir as well.

[jdkent]

Thanks @debottamKundu for testing this, @chrisfilo @effigies, is there a difference between output_spec().get() and _outputs().get()? I can't find a reason to use one versus the other.

[effigies]

@jdkent I don't know of a reason to choose one or the other. I guess output_spec() is less likely to have anything done behind your back, but they're probably both fine.

@debottamKundu Yes, you're right it should be 'out_dir', but that's not actually an error in the existing code.

[jdkent]

I added the output_spec().get() line to ICA_AROMA.py for the problem you guys found. I wasn't able to replicate your (@debottamKundu) other error after adding the fix, so maybe you could test it again after this pull request is merged?

[debottamKundu]

Okay I'll test it after the request is merged.