Tractography Data Format

[francopestilli]

It would be terrific to start a conversation about an agreed-upon data format for tractography.
@arokem @Garyfallidis

[mrneont]

Hi-

I am interested in this. I have worked on the tractography tools in AFNI (with Ziad Saad).

I imagine FSL developers would be interested, as would Frank Yeh @frankyeh of DSI-Studio and the developers of MRtrix3.

Thanks,
Paul Taylor

[francopestilli]

Yep! We should have this as an open discussion.

[frankyeh]

Love to see a new format standard. TRK file has been given me a lot of headaches and limited a lot of possible extension. DSI Studio will surely support any open standard for tractography.

[effigies]

While I don't necessarily think that a new tractography data format must be constrained by the NiBabel API, @MarcCote did put a fair bit of time and thought into the streamlines API. It might be worth thinking through whether this API is sufficient or if it's missing something.

Part of my thinking is that once a sufficient API is settled on, we can turn that around to quickly prototype more-or-less efficient implementations.

For what it's worth, I recently explored the API a bit and tried to summarize a bit in the following:

https://github.com/effigies/nh2020-nibabel/blob/ef3addf947004ca8f5610f34e767a578c4934c09/NiBabel.py#L821-L911

[MarcCote]

I totally agree with @effigies. Adding more people to the discussion. @frheault @jchoude

[francopestilli]

@MarcCote @frheault @jchoude @frankyeh great!
@effigies loading (partial data loading) and efficiency will be critical for the format.

[mrneont]

Trying to include a few other potentially interested people: @bjeurissen @jdtournier @neurolabusc
(Not finding an obvious FSL-tracking contact via github ID-- could someone else please help with that?)

[mrneont]

@effigies -- would be great to try that API with a demo.

Some functionality we value is keeping track of tracts as bundles-- if we put in some N>2 targets, we often care about any pairwise connections amongst those as separate bundles, because we are basically using tractography to parcellate the WM skeleton. Does that labeling/identifying of groups of tracts exist there?

[effigies]

The FMRIB contacts I know are @eduff and @pauldmccarthy. They might be able to point us to the right people...

[neurolabusc]

Before creating a new format, it might be worth considering the existing formats and see if any could be enhanced or improved, similar to the way that NIfTI maintained Analyze compatibility while directly addressing the weaknesses. The popular formats seem to be:

• BFloat and Bfloat.gz used by Camino.
• MRI studio's DAT format.
• PDB format is used by CINCH and DTI-Query.
• TCK format is used by MRTrix
• TRACT format used by AFNI.
• TRK format is used by TrackVis as well as dsi studio's compressed TRK.GZ. A nice feature is that this format includes support for bundles as properties (e.g. distinct bundles of tracks) as well as scalars (e.g. different z-scores along the length of a single track).
• Legacy VTK format (which have the filename extension .FIB or .VTK).

Unlike triangulated meshes, tractography can not benefit from indexing and stripping, so the existing formats all seem to be pretty similar (all describe node-to-node straight lines, not splines).

I concur with @mrneont that it is nice to have the ability to describe tracks as bundles. I think this makes TRK the most attractive choice (some other formats are not well documented, so they may support this feature).

Perhaps @frankyeh can expand on his frustrations with TRK. What are the limitations? Likewise, perhaps everyone (including @francopestilli who started this thread) can provide a wish list for desired features.

[Garyfallidis]

Hi all,

And thank you for bringing this up. I have to say this a recurrent topic. Every one or two years this re-emerges.

I suggest before you do anything else study what is already developed in the existing API in nibabel.

Marc-Alex and myself worked quite a bit to support all the basic needs. Accessing bundles fast, adding properties etc. Actually we have already implemented a fast version that can load/save tracks to npz (a numpy ) format. Which you can use if you have big data.

For me the main decision which requires feedback from the community is the formatting technology. Do you want to save the end result using json, hdf5, gITF or something else? If we can decide on that. Then we are set. The work to study previous formats is already mostly done at least on my side.

Nonetheless, see also a recent paper for a new format called Trako
https://arxiv.org/pdf/2004.13630.pdf

[frheault]

It is important to mention that no matter the file format, the main problems when it comes to standard will remain.
Most people have a lot of trouble with TRK because you can mess up the space or the header or both.
But the same remains true for vtk/tck, one could always write the TCK wrong (data) and/or provide the wrong nifti as a reference for the transformation.

No matter the new format, the same difficulties will remain. There is a thousand ways to write a TRK wrong, but many write it wrong and read it wrong too and it can work in their software. I think I was added due to my contribution to Dipy (StatefulTractogram). I think that no matter the new format, as long as people can have header attributes such as:

• Space (VOX, VOXMM, RASMM)
• Origin (of voxel, CORNER, CENTER)
• Affine (vox2rasmm or inverse)
• dimensions (shape of the diffusion from which the tractogram was computed)
• Voxel size (For verification with the affine)
• Voxel order (For verification with the affine)

I think I will be happy. For example, in my own code I used @MarcCote API to write an HDF5 format in which the length, offset, and data of one or multiple 'tractograms' are saved. So I can easily read any of these tractograms (I use it for connectomics, also could be used for bundles) and one could achieve the same to read any streamlines in particular. But as long as the attribute listed earlier are available anything can be done after that.

Also, if a new format is added to dipy. If it is statefulTractogram friendly it can easily be converted back to other commonly supported formats (TCK, TRK, VTK/FIB, DPY). If these efforts are made to have more efficient reading for computation, I think there is no problem with supporting more format. If the goal is to reduce confusion in how to write/read the format, I believe that a new format would not/never help. The unstructured nature of tractogram (not grid-like) makes it harder since the header and data are not fixed together when it comes to the spatial coherence.

PS : I personally think TRK is fine, everything is in the header. The problem is the variety of ways people can write/read it wrong. Making support across tools and labs pretty difficult. However, I think a strict approach to reading/write in dipy was beneficial on the long term. Short term, sure maybe half of the user probably hate me on some level, but a think a strict TRK (or at least a TCK always with the nifti that generated it) is superior than a lot of format. Just in term of available info, not for large scale computation visualisation and fancy reading/writing.

[arokem]

At the danger of somewhat rerouting the conversation, I guess since this has come up, this might also be a good time to discuss whether it would be beneficial to "upstream" the StatefulTractogram that @frheault has implemented in DIPY into nibabel. It's been "incubating" in DIPY since April 2019 (dipy/dipy#1812), with a few fixes and improvements along the way (e.g., dipy/dipy#2013, dipy/dipy#1997) that have followed rather closely the continuous improvements that have happened in nibabel. When Francois originally implemented this, we discussed the possibility of eventually moving the SFT object implementation here (see @effigies comment here about that: dipy/dipy#1812 (comment)). As someone who has been using SFT quite extensively in my own work, I find it rather helpful (where previously I was struggling with all the things that @frheault mentioned above). So I think that there could be a broader use for it. Just to echo @frheault's comment: a benefit of that would be that you could move between SFT-compliant formats with less fear of messing things up. I guess the question is one of timing and of potential future evolution of the SFT object. What are your thoughts, @frheault, (and others, of course)?

And to bring the discussion back to its start -- @francopestilli -- I am curious to hear: what needs are not currently addressed and prompted your original post here? Are you looking to store information that is not in the currently-supported formats? Or is there some limitation on performance that needs to be addressed?

[arokem]

Oh - just saw the mailing list messages and now understand where this originated (here: https://mail.python.org/pipermail/neuroimaging/2020-July/002161.html, and in other parts of that thread). Sorry: hard to stay on top of everything...

[frankyeh]

Oh - just saw the mailing list messages and now understand where this originated (here: https://mail.python.org/pipermail/neuroimaging/2020-July/002161.html, and in other parts of that thread). Sorry: hard to stay on top of everything...

The 10-million tractogram mentioned in that original thread is quite a challenge, and I am not sure if a new file format would be the answer. Emanuele already achieved 1 min loading time with TRK (pretty impressive).

…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.