Predicting Semantic Concepts in Heterogeneous Radiographic Images

Concept detection in a radiological image involves the identification of biomedical semantic entities within the given image. However, different modalities of radiological images make it difficult to design a single suitable approach that can handle this heterogeneity. Such imaging data also suffers from the problem of underrepresented sparse concepts where several concepts are present in very few training images making it difficult to train machine learning models to correctly predict their occurrence. This paper proposes a hierarchical approach for concept detection in radiological images using deep features extracted from the layers of convolutional neural networks. At the first level, the modality of the radiological image is identified. The second level of classification detects concepts present in the input image using multi-label classification by considering only those concepts that are relevant to images belonging to the same modality as the input image. This multi-label classification is performed using suitable classifiers that efficiently handle underrepresented sparse concepts. The proposed hierarchical approach for concept detection in radiological images outperforms state-of-the-art methods for different datasets.