Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Note why we don't implement TFCE in NiMARE (currently) #680

Merged
merged 3 commits into from
May 12, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions docs/cbma.rst
Original file line number Diff line number Diff line change
Expand Up @@ -138,3 +138,20 @@ The Monte Carlo FWE correction approach implemented in NiMARE produces three new
the voxel's summary statistic value lands on this null distribution.
**Voxel-level correction is generally more conservative than cluster-level correction,
so it is only recommended for very large meta-analyses (i.e., hundreds of studies).**

.. admonition:: What about threshold-free cluster enhancement?

TFCE :footcite:p:`smith2009threshold` is a voxel-level metric that combines signal magnitude and
cluster extent to enhance the importance of clusters that are large, have high magnitude, or both.

It can be applied to coordinate-based meta-analyses as an alternate metric to the
maximum summary statistic (``level-voxel``), cluster mass (``desc-mass``), or cluster size (``desc-size``).
However, recent work by Frahm et al. :footcite:p:`frahm2022evaluation` has indicated that the costs of performing
TFCE-based inference (e.g., massively increased computation time) outweigh any observable benefits.
As such, we have chosen not to implement TFCE-based correction within NiMARE,
although there is a closed pull request with an implementation that worked at the time it was closed
(see `#655 <https://github.com/neurostuff/NiMARE/pull/655>`_).

References
----------
.. footbibliography::
27 changes: 27 additions & 0 deletions docs/references.bib
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,20 @@ @article{fisher1946statistical
publisher={Oliver and Boyd}
}

@article{frahm2022evaluation,
author={Frahm, Lennart and Cieslik, Edna C. and Hoffstaedter, Felix and Satterthwaite, Theodore D. and Fox, Peter T. and Langner, Robert and Eickhoff, Simon B.},
title={Evaluation of thresholding methods for activation likelihood estimation meta-analysis via large-scale simulations},
journal={Human Brain Mapping},
volume={n/a},
number={n/a},
pages={},
year={2022},
keywords={family-wise error, FWE, multiple comparison correction, neuroimaging meta-analysis, significance thresholding, threshold-free cluster enhancement cluster extent},
doi={10.1002/hbm.25898},
url={https://onlinelibrary.wiley.com/doi/abs/10.1002/hbm.25898},
abstract={In recent neuroimaging studies, threshold-free cluster enhancement (TFCE) gained popularity as a sophisticated thresholding method for statistical inference. It was shown to feature higher sensitivity than the frequently used approach of controlling the cluster-level family-wise error (cFWE) and it does not require setting a cluster-forming threshold at voxel level. Here, we examined the applicability of TFCE to a widely used method for coordinate-based neuroimaging meta-analysis, Activation Likelihood Estimation (ALE), by means of large-scale simulations. We created over 200,000 artificial meta-analysis datasets by independently varying the total number of experiments included and the amount of spatial convergence across experiments. Next, we applied ALE to all datasets and compared the performance of TFCE to both voxel-level and cluster-level FWE correction approaches. All three multiple-comparison correction methods yielded valid results, with only about 5\% of the significant clusters being based on spurious convergence, which corresponds to the nominal level the methods were controlling for. On average, TFCE's sensitivity was comparable to that of cFWE correction, but it was slightly worse for a subset of parameter combinations, even after TFCE parameter optimization. cFWE yielded the largest significant clusters, closely followed by TFCE, while voxel-level FWE correction yielded substantially smaller clusters, showcasing its high spatial specificity. Given that TFCE does not outperform the standard cFWE correction but is computationally much more expensive, we conclude that employing TFCE for ALE cannot be recommended to the general user.}
}

@article{freedman1983nonstochastic,
title={A nonstochastic interpretation of reported significance levels},
author={Freedman, David and Lane, David},
Expand Down Expand Up @@ -230,6 +244,19 @@ @article{rubin2017decoding
doi={10.1371/journal.pcbi.1005649}
}

@article{smith2009threshold,
title={Threshold-free cluster enhancement: addressing problems of smoothing, threshold dependence and localisation in cluster inference},
author={Smith, Stephen M and Nichols, Thomas E},
journal={Neuroimage},
volume={44},
number={1},
pages={83--98},
year={2009},
publisher={Elsevier},
url={https://doi.org/10.1016/j.neuroimage.2008.03.061},
doi={10.1016/j.neuroimage.2008.03.061}
}

@misc{sochat2015ttoz,
author = {Sochat, Vanessa},
title = {TtoZ Original Release},
Expand Down