An Open Benchmark Challenge for Motion Correction of Myocardial Perfusion MRI

Cardiac magnetic resonance perfusion examinations enable noninvasive quantification of myocardial blood flow. However, motion between frames due to breathing must be corrected for quantitative analysis. Although several methods have been proposed, there is a lack of widely available benchmarks to compare different algorithms. We sought to compare many algorithms from several groups in an open benchmark challenge. Nine clinical studies from two different centers comprising normal and diseased myocardium at both rest and stress were made available for this study. The primary validation measure was regional myocardial blood flow based on the transfer coefficient
$(K^{\rm{trans}})$
, which was computed using a compartment model and the myocardial perfusion reserve (MPR) index. The ground truth was calculated using contours drawn manually on all frames by a single observer, and visually inspected by a second observer. Six groups participated and 19 different motion correction algorithms were compared. Each method used one of three different motion models: rigid, global affine, or local deformation. The similarity metric also varied with methods employing either sum-of-squared differences, mutual information, or cross correlation. There were no significant differences in
$K^{\rm{trans}}$
or MPR compared across different motion models or similarity metrics. Compared with the ground truth, only
$K^{\rm{trans}}$
for the sum-of-squared differences metric, and for local deformation motion models, had significant bias. In conclusion, the open benchmark enabled evaluation of clinical perfusion indices over a wide range of methods. In particular, there was no benefit of nonrigid registration techniques over the other methods evaluated in this study. The benchmark data and results are available from the Cardiac Atlas Project (
www.cardiacatlas.org
).