Enhancing Road Safety: A Comprehensive Driver Behavior Scoring Framework with K-Means Action Segmentation and Deep Learning Behavior Detection

The rising concern for road safety due to increasing vehicle numbers requires innovative strategies to address unsafe driving practices. This paper presents a comprehensive Driver Behavior Scoring Framework aimed at improving road safety. It consists of five modules: Video Segmentation, Behavior Classification, Voice and Speech Analysis, Facial Analysis, and Safety Scoring. In Video Segmentation, we propose a novel approach using VGG-16 and k-means for action segmentation. The Behavior Classification module utilizes a CNN-LSTM classifier, which obtained an F1-score of 87.5%. Voice and Speech Analysis use a DistilBERT-based sentiment analysis achieving 93.42% accuracy in sentiment detection and 76.6% accuracy in profanity detection. Facial Analysis detects eye and mouth movements with MAE of 0 and 1.6 respectively, and identifies anger and happiness behaviors with an MAE of 44 using a CNN-based pretrained model. The fifth module introduces a Linear Regression-based safety scoring algorithm trained on survey data, yielding an RMSE of 1.1616.