Going Deep in Medical Image Analysis: Concepts, Methods, Challenges, and Future Directions

Medical image analysis is currently experiencing a paradigm shift due to deep learning. This technology has recently attracted so much interest of the Medical Imaging Community that it led to a specialized conference in “Medical Imaging with Deep Learning” in the year 2018. This paper surveys the recent developments in this direction and provides a critical review of the related major aspects. We organize the reviewed literature according to the underlying pattern recognition tasks and further sub-categorize it following a taxonomy based on human anatomy. This paper does not assume prior knowledge of deep learning and makes a significant contribution in explaining the core deep learning concepts to the non-experts in the Medical Community. This paper provides a unique computer vision/machine learning perspective taken on the advances of deep learning in medical imaging. This enables us to single out “lack of appropriately annotated large-scale data sets” as the core challenge (among other challenges) in this research direction. We draw on the insights from the sister research fields of computer vision, pattern recognition, and machine learning, where the techniques of dealing with such challenges have already matured, to provide promising directions for the Medical Imaging Community to fully harness deep learning in the future.