A Hybrid Approach for Incorporating Deep Visual Features and Side Channel Information with Applications to AMD Detection

This work investigates a hybrid method based on random forests and deep image features to combine non-visual side channel information with image data for classification. We apply this to automated retinal image analysis (ARIA) and the detection of age-related macular degeneration (AMD). For evaluation, we use a dataset collected by the National Institute of Health with over 4000 study participants. The non-visual side channel data includes information related to demographics (e.g. ethnicity), lifestyle (e.g. sunlight exposure), and prior conditions (e.g. cataracts). Our study, which compares the performance of different feature combinations, offers preliminary results that constitute a baseline for future investigations on joint deep visual and side channel feature exploitation for AMD detection. This approach could potentially be used for other medical image analysis problems.