A Deep Temporal Model for Mental Fatigue Detection

Fatigue is not only common in clinical patients but is also prevalent in the healthy population. A lot of studies have been done to detect mental fatigue. Most of the current fatigue detection systems are based on individual questionnaire and computer vision. In this paper, we proposed a deep temporal model named Deep Convolutional Autoencoding Memory network to detect mental fatigue in real time based on biometrics (e.g., GSR, HR, R-R intervals and skin temperature) collected with smart band. The model can be regarded as a One-Class classifier consisted of two sub-networks: Characterization Network and Memory Network that can be co-optimized to extract features from raw data automatically and generate baseline model which can describe the sequential patterns of non-fatigue data. The major contributions of this paper are as follows: First, we solved the class imbalance problem between normal data and fatigue data through performing One-Class classification; Second, Characterization Network can extract features from the heterogeneous biometrics automatically; Third, Memory Network can generate a temporal model which accurately describes the normal pattern of sequential data; Last, our model uses a novel decision criterion to make the detection decision. Performance of the proposed method is validated on a realistic environmental dataset containing the biometrics of 6 participants collected in 6 weeks. Experimental results show that the accuracy of fatigue detection attains 82.9%, which is superior to the representative off-the-shelf methods.