Multimodal Fusion for Dementia Detection using Voice and Facial Features

Early detection of dementia is important in the aging society. Research has shown that early diagnosis and treatment can effectively slow down cognitive decline in the elderly. Our goal is to develop a low-cost dementia detection method by analyzing videos capturing the progress of potential patients taking the Short Portable Mental Status Questionnaire (SPMSQ). We propose a multimodal fusion method that effectively predicts the degree of dementia based on both voice and facial features. Because of lacking enough training data, we propose a feature augmentation method based on the mix-up technique to train a more effective model. Experimental results demonstrate the effectiveness of the proposed method.