string being treated as list in prepare_epi_matching_wf

[bkraft4257]

I have trying to us fmriprep-docker on a data set with TOPUP (AP, PA) fieldmap correction. I have run bids-validator on my data set (single subject at this moment) and everything looks ok. My json files for the two AP and PA acquisitions are

>> cat sub-34P1030fmap_acq-rest_dir-AP_epi.json
 {
 "PhaseEncodingDirection": "j-",
 "TotalReadoutTime": 0.079,
 "IntendedFor": "func/sub-34P1030fmap_task-rest_bold.nii.gz"
}

and

>> cat sub-34P1030fmap_acq-rest_dir-AP_epi.json
 {
 "PhaseEncodingDirection": "j-",
 "TotalReadoutTime": 0.079,
 "IntendedFor": "func/sub-34P1030fmap_task-rest_bold.nii.gz"
 }

respectively. Unfortunately, I encounter a problem very quickly into the processing. The crash report in the HTML report is at the end of this post. It looks as if the input file for the merge node is not being treated as a list but as a string. This results in the filename being split into individual characters. At least that is my interpretation of what is going on.

Any suggestions on how to fix this problem? Basically, I just want to run a functional data set with a TOPUP distortion correction.

Thanks in advance for your help.

Bob

Node Name: fmriprep_wf.single_subject_34P1030_wf.func_preproc_task_rest_wf.pepolar_unwarp_wf.prepare_epi_matching_wf.merge
File: crash-20170721-203045-root-merge-472715c2-90c1-43cc-b840-034b1a4937ee.pklz
Working Directory: /scratch/fmriprep_wf/single_subject_34P1030_wf/func_preproc_task_rest_wf/pepolar_unwarp_wf/prepare_epi_matching_wf/merge
Inputs: 
args:
auto_detect_sensitivity: True
average_metric:
environ: {'SUBJECTS_DIR': '/opt/freesurfer/subjects'}
fixed_timepoint: True
ignore_exception: False
in_files:
in_intensity_scales:
initial_timepoint: 1
initial_transforms:
intensity_scaling: True
no_iteration: True
out_file: template.nii.gz
outlier_sensitivity:
scaled_intensity_outputs:
subjects_dir: /opt/freesurfer/subjects
subsample_threshold: 200
terminal_output: stream
transform_outputs:
Traceback (most recent call last):
File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/plugins/multiproc.py", line 301, in sendprocs_to_workers
jobid].hash_exists()
File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 244, in hash_exists
hashed_inputs, hashvalue = self.gethashval()
File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 407, in gethashval
self.getinputs()
File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 467, in getinputs
self.set_input(key, deepcopy(output_value))
File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 223, in set_input
setattr(self.inputs, parameter, deepcopy(val))
File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base.py", line 2076, in validate
value = super(MultiPath, self).validate(object, name, newvalue)
File "/usr/local/miniconda/lib/python3.6/site-packages/traits/trait_types.py", line 2336, in validate
return TraitListObject( self, object, name, value )
File "/usr/local/miniconda/lib/python3.6/site-packages/traits/trait_handlers.py", line 2313, in init
raise excp
File "/usr/local/miniconda/lib/python3.6/site-packages/traits/trait_handlers.py", line 2305, in init
value = [ validate( object, name, val ) for val in value ]
File "/usr/local/miniconda/lib/python3.6/site-packages/traits/trait_handlers.py", line 2305, in 
value = [ validate( object, name, val ) for val in value ]
File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/traits_extension.py", line 92, in validate
self.info_text, value))
traits.trait_errors.TraitError: The trait 'in_files' of a RobustTemplateInputSpec instance is an existing file name, but the path '/' does not exist.
Error setting node input:
Node: merge
input: in_files
results_file: /scratch/fmriprep_wf/single_subject_34P1030_wf/func_preproc_task_rest_wf/pepolar_unwarp_wf/prepare_epi_matching_wf/split/result_split.pklz
value: ['/', 's', 'c', 'r', 'a', 't', 'c', 'h', '/', 'f', 'm', 'r', 'i', 'p', 'r', 'e', 'p', '_', 'w', 'f', '/', 's', 'i', 'n', 'g', 'l', 'e', '_', 's', 'u', 'b', 'j', 'e', 'c', 't', '_', '3', '4', 'P', '1', '0', '3', '0', '_', 'w', 'f', '/', 'f', 'u', 'n', 'c', '_', 'p', 'r', 'e', 'p', 'r', 'o', 'c', '_', 't', 'a', 's', 'k', '_', 'r', 'e', 's', 't', '_', 'w', 'f', '/', 'p', 'e', 'p', 'o', 'l', 'a', 'r', '_', 'u', 'n', 'w', 'a', 'r', 'p', '_', 'w', 'f', '/', 'p', 'r', 'e', 'p', 'a', 'r', 'e', '_', 'e', 'p', 'i', '_', 'm', 'a', 't', 'c', 'h', 'i', 'n', 'g', '_', 'w', 'f', '/', 's', 'p', 'l', 'i', 't', '/', 'm', 'a', 'p', 'f', 'l', 'o', 'w', '/', '_', 's', 'p', 'l', 'i', 't', '0', '/', 'v', 'o', 'l', '0', '0', '0', '0', '.', 'n', 'i', 'i', '.', 'g', 'z']

[effigies]

Thanks for reporting. I think we need to check to make sure that paths are being treated properly throughout this portion of the pipeline, but first we should just check whether @chrisfilo's intuition was correct:

Can you make the "IntendedFor" field read:

"IntendedFor": [ "func/sub-34P1030fmap_task-rest_bold.nii.gz" ]

[bkraft4257]

Unfortunately, this simple modification of the json file did not solve the problem. Here is a copy of the pypeline.log.txt.

There are other errors in the log that occur earlier. I don't know if they are related.

>> log grep -i error pypeline.log
170724-17:38:01,888 workflow ERROR:
170724-17:38:01,888 workflow ERROR:
    raise ValueError(msg)
ValueError: DerivativesDataSink requires a value for input 'in_file'. For a list of required inputs, see DerivativesDataSink.help()
    raise ValueError(msg)
ValueError: DerivativesDataSink requires a value for input 'in_file'. For a list of required inputs, see DerivativesDataSink.help()
170724-17:38:02,225 workflow ERROR:
170724-17:38:02,225 workflow ERROR:
    raise ValueError(msg)
ValueError: DerivativesDataSink requires a value for input 'in_file'. For a list of required inputs, see DerivativesDataSink.help()
    raise ValueError(msg)
ValueError: DerivativesDataSink requires a value for input 'in_file'. For a list of required inputs, see DerivativesDataSink.help()
170724-17:38:03,342 workflow ERROR:
170724-17:38:03,342 workflow ERROR:
traits.trait_errors.TraitError: The trait 'in_files' of a RobustTemplateInputSpec instance is an existing file name, but the path  '/' does not exist.
Error setting node input:
traits.trait_errors.TraitError: The trait 'in_files' of a RobustTemplateInputSpec instance is an existing file name, but the path  '/' does not exist.
Error setting node input:

Please let me know if you need anything else.

pypeline.log.txt

[effigies]

Hi Bob, I'm not seeing any errors in the log file you posted. In fact, it seems to have still been running when you uploaded it:

...
170724-17:44:00,572 workflow DEBUG:
	 MultiProcPlugin timeout before signal received. Deadlock averted??
170724-17:44:00,572 workflow DEBUG:
	 MultiProcPlugin timeout before signal received. Deadlock averted??
170724-17:44:00,574 workflow DEBUG:
	 Number of pending tasks: 1
170724-17:44:00,574 workflow DEBUG:
	 Number of pending tasks: 1

If it did crash again, could you copy-paste the traceback? Otherwise, it may just be that you need to wait on some of the slower parts of the job.

[bkraft4257]

My apologies. I posted the wrong log file. Please look at this file.
pypeline.log.txt

[effigies]

Ah, that looks more like our issue. I believe the problem is here. Thanks for the report.

You also seem to be running up against the same DataSink-related issue as in #595, so that's another thing to look at.

[bkraft4257]

Thanks for figuring it out so quickly. So what happens next?

Also issue #595 is a "set nipype crashfile format as plain text" not a DataSink related issue. What am I missing?

I apologize for my ignorance but I have only been using fmriprep for 15 hours.

[effigies]

I'll submit a fix tomorrow (didn't get to it, today). Assuming everybody's happy with it, it'll get merged. Once the current outstanding bugs are fixed, I'll draft a new release. I hope to have that out by the end of the week.

Regarding #595, there's an error that showed up in both of your reports, though it wasn't the subject of either issue.

[effigies]

Hi Bob, could you paste the contents of fmriprep_wf/single_subject_34P1030_wf/func_preproc_task_rest_wf/pepolar_unwarp_wf/prepare_epi_matching_wf/split/_report/report.rst?

This will be in whatever directory you were passing with the -w flag.