Real-Time Deep Neuro-Vision Embedded Processing System for Saliency-based Car Driving Safety Monitoring

Recently, much interest has been aroused by scientific community regarding visual saliency-based applications. The proposed approach contributes to the progressive growth of knowledge of video saliency applications in the automotive field. Through the visual saliency detection, the car driver assistance systems (ADAS i.e. Advanced Driver Assistance Systems) are able to process the driving observation scene selectively. Specifically, it has been observed that the drivers along the driving route, focuses his/her gaze on some objects rather than others. This is determined by the perceptual activity of the brain which through the visual saliency determine the focused scene. We propose a driving safety assessment pipeline which combines a near-real time drowsiness car driver monitoring system driven by a visual saliency detection applied to the acquired driving scene. The proposed approach includes ad-hoc 3D pre-trained Semantic Segmentation Deep Network combined with ad-hoc 1D temporal Deep Dilated Convolutional Neural Network. This architecture was developed for the embedded platform based on STA1295 Accordo5 core (ARM A7 Dual-Cores) embedding an hardware graphics accelerator. The proposed system embeds a bio-sensor which will be placed on the steering wheel of the car having the target to collect the driver's Photoplethysmography (PPG) signal and which will result in a control of driver attention level. The so collected PPG time-series will be classified by the mentioned 1D Temporal Deep Convolutional Network which provides an assessment of the driver attention level. A final analyzer block verifies if the car driver attention level is adequate for the saliency-based scene classification. The performed tests confirmed the effectiveness of the overall proposed pipeline.