Using Convolutional Neural Network to Automate ACR MRI Low-Contrast Detectability Test

According to the American College of Radiology (ACR), the performance of magnetic resonance imaging (MRI) scanners should be monitored using phantom images acquired weekly. These quality assurance images are usually analyzed by a technician, but automated analysis has been proposed to reduce costs and improve repeatability. Reports on the automation of low-contrast detectability tests are scarce, and none can completely replace human work. In previous works, we have demonstrated that machine learning methods can be used to learn the subtleties of image quality and the visual assessment of technicians. We showed that machines are able to mimic human perception quite accurately. In these works, we used hand-designed image quality features. In the present work, we use a deep learning method to automatically design appropriate image features. By training this network on a large base with visual assessments from multiple technicians, we show that the machine can be taught to assess MRI image quality better than any technician alone, justifying its widespread adoption. Our dataset contained 12,000 binary responses to the detectability of low-contrast structures (“holes”). We used the median of the technicians’ responses as the gold standard. To increase statistical power, we repeated training and testing 5 times, using 5-fold cross-validation. We obtained a mean AUC (area under the ROC curve) of 0.983± 0.003. At the point of equal error rate, the mean accuracy, sensitivity and specificity were 93.2±0.7%, numbers higher than those achieved by any technician alone. Applying the obtained model to a completely independent test dataset with 10,800 structures, we obtained an AUC of 0.979. The predictions of our model in classifying spokes (sets of 3 holes) agree in 93.83% of the cases with the median of the responses of the technicians. These results again are better than the responses of any individual technician. We conclude that the ACR test can be performed by a machine with greater reliability than individual technicians.