A Path Signature Approach for Speech-Based Dementia Detection

People who have dementia show a decline in their speech abilities. In speech-based dementia detection, the difficulty has remained the representation of an individual's sequential temporal variation of speech is related to dementia symptoms with fix-length features. In this letter, a novel feature extraction method is proposed for extracting fix-length features from unfixed-length audio recordings for dementia detection. When diagnosing dementia, an automatic speech recognition (ASR) system is necessary for extracting linguistic information when constructing an automatic dementia detection system. This letter uses wav2vec2.0, a self-supervised end-to-end ASR system, to achieve such a goal. Similar to the pipeline ASR system, which has been used for extracting the sequential speak-and-pause patterns related to dementia using estimated time alignment information, we propose using character-level transcripts to extract speak-and-pause patterns. Path signature technology, which can represent a sequential feature with a trajectory in the un-parameterised path space, is proposed to describe speak-and-pause patterns embedded in character-level transcripts into character path signatures. Similarly, the variable-length embedding matrices extracted from wav2vec2.0’s contextual layers are also represented with their acoustic path signatures. The experiments are designed based on three publicly available datasets: DementiaBank, ADReSS and ADReSSo. The results show that: (1). The distinguished information embedded in the character path signature is visualised for dementia detection; (2). The acoustic path signature and character path signature individually can show superior performance on all three publicly available datasets. (3). Combining the character path signature with the acoustic path signature can considerably increase performance over the ADReSSo dataset.