Estimating Depression Severity from Long-Sequence Face Videos via an Ensemble Global Diverse Convolutional Model

Depression is a mood disorder that has serious consequences for both individuals and society. The current diagnosis of depression relies on questionnaires and clinical interviews, which are complex and subjective, as they depend on multiple factors such as the patient’s comorbidities, cognitive ability, honesty in describing symptoms, and the experience and motivation of the clinician. An automated way of estimating depression severity objectively would, therefore, be of great assistance to clinicians. Over the last decade, various affective computing systems have been proposed that use machine learning, speech analysis, and computer vision techniques to extract unimodal or multimodal cues and estimate the severity of depression. Temporal information plays a crucial role in learning spatiotemporal patterns in depression data. When inferring depression severity from face videos, analysing long sequences at different temporal scales is potentially more informative than analysing short ones. Therefore, a new approach based on long sequence structured global convolution and various temporal scales on diverse kernels in an end-to-end ensemble model has been developed and evaluated for estimating depression severity from face video data. The application of this long-range dependency technique is novel in automated depression analysis. The proposed ensemble model explores the role of temporal scales in assessing depression severity from facial movement information and outperforms common deep learning models and single structured global convolution models. The results confirm that analysing facial movements at different temporal scales is an important component towards effective diagnostic aids in depression analysis.