Multi-label Modality Classification for Figures in Biomedical Literature

The figures found in biomedical literature are a vital part of biomedical research, education and clinical decision. The multitude of their modalities and the lack of corresponding meta-data, constitute search and information retrieval a difficult task. We present multi-label modality classification approaches for biomedical figures. In particular, we investigate using both simple and compound figures for training a multi-label model to be used for annotating either all figures, or only those predicted as compound by an initial compound figure detection model. Using data from the medical task of ImageCLEF 2016, we train our approaches with visual features and compare them with the standard approach involving compound figure separation into sub-figures. Furthermore, we present a web application for medical figure retrieval, which is based on one of our classification approaches and allows users to search for figures of PubMed Central.