Moving beyond processing- and analysis-related variation in resting-state functional brain imaging

Xinhui Li; Nathalia Bianchini Esper; Lei Ai; Steve Giavasis; Hecheng Jin; Eric Feczko; Ting Xu; Jon Clucas; Alexandre Franco; Anibal Sólon Heinsfeld; Azeez Adebimpe; Joshua T Vogelstein; Chao-Gan Yan; Oscar Esteban; Russell A Poldrack; Cameron Craddock; Damien Fair; Theodore Satterthwaite; Gregory Kiar; Michael P Milham

doi:10.1038/s41562-024-01942-4

Moving beyond processing- and analysis-related variation in resting-state functional brain imaging

Nat Hum Behav. 2024 Aug 5. doi: 10.1038/s41562-024-01942-4. Online ahead of print.

Authors

Xinhui Li^#¹, Nathalia Bianchini Esper^#¹, Lei Ai¹, Steve Giavasis¹, Hecheng Jin¹, Eric Feczko^{2

3

4

5}, Ting Xu¹, Jon Clucas¹, Alexandre Franco^{1

6}, Anibal Sólon Heinsfeld⁷, Azeez Adebimpe⁸, Joshua T Vogelstein⁹, Chao-Gan Yan^{10

11

12

13}, Oscar Esteban^{14

15}, Russell A Poldrack¹⁵, Cameron Craddock⁷, Damien Fair^{2

3

5

16}, Theodore Satterthwaite^{8

17}, Gregory Kiar^#¹, Michael P Milham^#^{18

19}

Affiliations

¹ Child Mind Institute, New York, NY, USA.
² Masonic Institute for the Developing Brain, University of Minnesota, Minneapolis, MN, USA.
³ Department of Pediatrics, University of Minnesota, Minneapolis, MN, USA.
⁴ Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, OR, USA.
⁵ Department of Psychiatry, Oregon Health and Science University, Portland, OR, USA.
⁶ Center for Biomedical Imaging and Neuromodulation, Nathan Kline Institute, Orangeburg, NY, USA.
⁷ Department of Diagnostic Medicine, The University of Texas at Austin, Austin, TX, USA.
⁸ Penn Lifespan Informatics and Neuroimaging Center, Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
⁹ Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA.
¹⁰ CAS Key Laboratory of Behavioral Science, Institute of Psychology, Beijing, China.
¹¹ Magnetic Resonance Imaging Research Center, Institute of Psychology, Chinese Academy of Sciences, Beijing, China.
¹² International Big-Data Center for Depression Research, Chinese Academy of Sciences, Beijing, China.
¹³ Department of Psychology, University of Chinese Academy of Sciences, Beijing, China.
¹⁴ Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland.
¹⁵ Department of Psychology, Stanford University, San Francisco, CA, USA.
¹⁶ Institute of Child Development, University of Minnesota, Minneapolis, MN, USA.
¹⁷ Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
¹⁸ Child Mind Institute, New York, NY, USA. michael.milham@childmind.org.
¹⁹ Center for Biomedical Imaging and Neuromodulation, Nathan Kline Institute, Orangeburg, NY, USA. michael.milham@childmind.org.

^# Contributed equally.

PMID: 39103610
DOI: 10.1038/s41562-024-01942-4

Abstract

When fields lack consensus standard methods and accessible ground truths, reproducibility can be more of an ideal than a reality. Such has been the case for functional neuroimaging, where there exists a sprawling space of tools and processing pipelines. We provide a critical evaluation of the impact of differences across five independently developed minimal preprocessing pipelines for functional magnetic resonance imaging. We show that, even when handling identical data, interpipeline agreement was only moderate, critically shedding light on a factor that limits cross-study reproducibility. We show that low interpipeline agreement can go unrecognized until the reliability of the underlying data is high, which is increasingly the case as the field progresses. Crucially we show that, when interpipeline agreement is compromised, so too is the consistency of insights from brain-wide association studies. We highlight the importance of comparing analytic configurations, because both widely discussed and commonly overlooked decisions can lead to marked variation.

Abstract

Grants and funding