Low-Power Hardware-Friendly Fatigued Driving Detection Based on Feature Fusion and Time Series Classification

The grim reality of fatigued driving-caused traffic accidents indicates the urgent need to establish a robust fatigued driving detection method to prevent drivers from falling asleep while driving. Previous studies on this topic have little consideration on the sequence-level characteristics of driver’s behaviors, furthermore, many of them only focus on one type of feature, which limits the accuracy and flexibility. Besides, most existing computer vision-based studies perform experiments on high-performance hardware like GPUs, which can not demonstrate their performance in the reality, as it is impractical to equip such high-end computing hardware in vehicles. This paper proposes a novel low-power hardware-friendly fatigued driving detection method based on multi-feature fusion and time series classification. The system utilizes a camera installed on the vehicle for driver monitoring. Continuous frames of a specific duration are a detection sequence. Multiple physical features including head orientation, face landmark coordinates, EFV (Eye Feature Vector), and MFV (Mouth Feature Vector) are extracted from each frame. A neural network optimized for low-power hardware is used to classify the multivariate time series samples denoting the driver’s state into two classes: fatigued or alert. Sequence-level analysis reduces the false positive rate by avoiding being affected by accidental symptoms. Multi-feature enhances the capability to give the right detection results in complex and changeable driving scenarios. The proposed method is comprehensively evaluated on an ARM platform-based device simulating the low-power in-vehicle hardware environment. Experiments show our models surpass existing studies by up to 11.49% on accuracy and achieve up to 1.41× speedup, proving the significant advantages of the proposed method.