ROC analysis of PET imaging performance of two PET/CT scanners based on lesion detectability in a torso phantom

Positron emission tomography (PET) and computed tomography (CT) are established imaging tools. Integrated PET/CT scanners are now commonly used in radiation treatment planning to delineate accurate tumor margins, to identify nodal involvement, and in post-treatment follow-up assessments. The use of PET/CT images for quantitative selection of dose prescriptions is also being considered. The usefulness of the images produced for these applications depends strongly on the interplay between the performance of the imaging system and the skill of the diagnostician. The purpose of this study was to evaluate the performance of the PET component of two PET/CT scanners using receiver operating characteristic (ROC) analysis of the ability of human observers to identify the presence of simulated tumors in PET images of an anthropomorphic torso phantom. We also assessed the effects of PET acquisition and reconstruction parameters (such as acquisition length and reconstruction method) on the observers' performances and the relationship of performance to physical system factors (such as resolution and sensitivity). Image data of ¹⁸F-fluorodeoxyglucose distribution in the phantom and water-filled acrylic spheres were acquired with the PET/CT scanners; the spheres simulate tumors. Data were obtained for a variety of acquisition and reconstruction parameters available on the scanners. Positive and negative images were generated from the acquired image data, representing up to two simulated tumors per image as visualized with the various system parameters. The images were rated for the presence of tumors by "trained" observers (physicians who use radiologic images daily in patient management) and "untrained" observers (medical physicists, post-doctoral researchers, and medical physics graduate students who do not use these images routinely). ROC curves were generated to assess readers' performances relative to the scanners' parameters as characterized by area under the curve (A_z). Overall, the untrained group provided little discriminating power in this study, perhaps because of its heterogeneous levels of experience. Trained observers consistently outperformed untrained observers. As well, trained observers performed significantly better using Reveal HD system images than with images generated in the Discovery ST two-dimensional (2D) mode (p < 0.01); however, differences between other pairings of scanners were not significant. Of the three reconstruction algorithms investigated (ordered-subset expectation maximization [OSEM], filtered backprojection, and direct-inversion Fourier transform), OSEM produced the smallest A_z for trained observers. Because of its higher sensitivity and higher resolution, the fully 3D Reveal HD facilitated better performance by the trained observers than the Discovery ST 2D mode. To further evaluate the use of PET/CT in radiation therapy treatment planning, we will plan to expand this study to include fused PET/CT images and specific radiation therapy treatment planning tasks, such as drawing clinical target volume margins around tumors.