Metadata updates required for pybids 0.7 compatibility

[effigies]

This is a thread to keep track of changes that need to be made to the metadata outputs of fMRIPrep to conform with pybids' interpretation of the BIDS Derivatives RC1 spec. I am running into these issues trying to update fitlins to work, using derivatives created with fMRIPrep 1.2.1.

cc @adelavega @tyarkoni As some of these may end up being pybids/spec issues.

File-level metadata

bold.json

It is no longer possible to refer back to the source file for metadata, such as RepetitionTime, so fMRIPrep will need to update old metadata files, removing invalid metadata and saving to the new output directory.

Note that this is going to need special handling for sparse acquisition paradigms. For instance, RepetitionTime and SliceTiming are a valid way to specify a sparse paradigm, but DelayTime will need to be calculated to ensure the same acquisition duration is calculated. (AcquisitionDuration, for some reason, is mutually exclusive with RepetitionTime.)

dataset_description.json

Resolved in bids-standard/pybids#295

[adelavega]

@chrisfilo Do you have any opinion as to whether the PipelineDescription should be a nested dictionary, or fields should be flat? That is, is there any chance we will update the derivatives spec or is pybids's interpretation correct/final?

Looks like we just need to make an executive decision on this. My vote would be to update the spec for it to be a nested dictionary, but I can live with either.

[effigies]

👍 to everything @adelavega said there.

[tyarkoni]

I don't see any good reason why metadata files should be constrained to be flat anywhere in the spec or pybids. IMO, if pybids currently disallows nesting, we should change that.

[effigies]

@tyarkoni It's specifically the interpretation of the "PipelineDescription.Name" syntax in the spec. Does that mean the field has a dot in it, or is it a shorthand for nested dictionaries? See the first section in the initial post.

[chrisgorgo]

It was intended to be a nested dictionary (like in PET proposal). We should add an example to the spec to clarify this.

[adelavega]

Okay great, so aside from clarifying the spec, all is needed is a PR to pybids to correct it's interpretation. No changes to fmriprep needed.

[effigies]

This one is becoming critical for FitLins usage, so we should figure out the best way to do it. There are basically two questions:

1. Which sidecar files do we create? One for each image, or as high in the directory structure as possible? The former is easier, but definitely more verbose; the latter is going to be harder to do right, especially as any differences across subjects/tasks/(any other selectable unit) may not be available to the individual run.

2. What metadata do we try to preserve? Just the mandatory metadata, or anything that we don't explicitly render invalid? I think it's pretty easy to do mandatory + things we have added ourselves (e.g., Space), and there are some things we know we'll invalidate (e.g., SliceTiming) but I'm a little worried that it will be hard to verify all metadata entries are still valid that are not explicitly invalidated.

[oesteban]

• Which sidecar files do we create? One for each image, or as high in the directory structure as possible? The former is easier, but definitely more verbose; the latter is going to be harder to do right, especially as any differences across subjects/tasks/(any other selectable unit) may not be available to the individual run.

With nipreps/niworkflows#318 merged, it will be trivial to do option 1). Not sure if 2) does not go beyond the participant level of analysis (although the pipeline description also trespasses that line and it is justified).

• What metadata do we try to preserve? Just the mandatory metadata, or anything that we don't explicitly render invalid? I think it's pretty easy to do mandatory + things we have added ourselves (e.g., Space), and there are some things we know we'll invalidate (e.g., SliceTiming) but I'm a little worried that it will be hard to verify all metadata entries are still valid that are not explicitly invalidated.

I think we should start with a white-list approach (mandatory + things we have added ourselves). The way nipreps/niworkflows#318 is implemented could also make this pretty convenient.