Despite their wide-spread use, only limited information is available on the comparative test-retest reliability of task-based functional and resting state magnetic resonance imaging measures of blood oxygen level dependence (tb-fMRI and rs-fMRI) and cerebral blood flow (CBF) using arterial spin labeling. This information is critical to designing properly powered longitudinal studies. Here we comprehensively quantified and compared the test-retest reliability and reproducibility performance of 8 commonly applied fMRI tasks, 6 rs-fMRI metrics and CBF in 30 healthy volunteers. We find large variability in test-retest reliability performance across the different tb-fMRI paradigms and rs-fMRI metrics, ranging from poor to excellent. A larger extent of activation in tb-fMRI is linked to higher between-subject reliability of the respective task suggesting that differences in the amount of activation may be used as a first reliability estimate of novel tb-fMRI paradigms. For rs-fMRI, a good reliability of local activity estimates is paralleled by poor performance of global connectivity metrics. Evaluated CBF measures provide in general a good to excellent test-reliability matching or surpassing the best performing tb-fMRI and rs-fMRI metrics. This comprehensive effort allows for direct comparisons of test-retest reliability between the evaluated MRI domains and measures to aid the design of future tb-fMRI, rs-fMRI and CBF studies.
