Computer Vision Based Vulnerable Road Users Hand Signal Recognition

With development of computer vision (CV) and Artificial Intelligent (AI) the domains like object detection, object tracking their development from 2D to 3D tracking and human pose estimation have gained a lot of popularity and demand in the field of Advanced Driver Assistance Systems (ADAS) and Autonomous Driving (AD). Basically, these systems are designed with an objective to increase the safety and avoid road accidents while driving. These systems develop the ability of car driver to react to various types of hazards and dangers on the road. If it is possible to know the intentions of Vulnerable Road Users (VRUs) like pedestrian, cyclist or motorbike riders then the probability of road accidents can be reduced to great extents. The ADAS systems are able to warn the car driver through automatic systems and thus provide high safety. This paper proposes Convolutional neural Networks (CNN) based VRUs hand signal recognition system based on Region Multi-Person Pose Estimation (RMPE) [1] framework for human pose estimation to know the hand signals given by the bicyclist, scooter, e-scooter and motorbike riders to warn the car driver. The VRUs are well aware of the traffic rules and regulations and also follow them. Based on the signal given by bicyclist the ego-vehicle can stop or reduce its speed. The focus of this novel algorithm proposed in this paper is that it is capable of recognizing all the six hand signals of bicyclists left, right, stop, give way, slow down and road hazard which are not addressed before in any previous works. Also, this algorithm can recognize arm signals of scooter, e-scooter riders and motorbike riders efficiently. The dataset used for testing is the self-captured dataset for VRUs Arm Signal Recognition and to generalize it Cyclist Arm Signal Recognition (CASR) [2] dataset is also used. The proposed work provides state-of-the-art (SOTA) results for VRUs hand signal recognition and frame rate of 10 FPS and 92% acuuracy.