Vehicle Behavior Recognition using Multi-Stream 3D Convolutional Neural Network

Vehicle behavior recognition take an essential part in the community of intelligent driving assistance systems. Recent approaches with 3D convolutional neural network (3DCNN) achieve reasonable recognition performance in laboratory settings. However, due to the complexity of network, the model is large and the reasoning is slow, so it is difficult to be applied in practice. In order to better handle the trade-off between size, precision and reasoning speed,a lightweight multi-stream 3DCNN model is proposed in this paper, which achieves fast reasoning speed and small model size while maintaining high precision. The 3DCNN model consists of three parts. Firstly, the module of SELayer-3DCNN is developed to extract appearance information from the RGB image sequence. The motion and edge information are also extracted from the optical flow sequence and edge image sequence, respectively. The edge information is applied to enhance the optical flow features. Secondly, a novel channel attention fusion strategy is proposed to improve the feature fusion and network ability. Finally, a 3D-RFB module is proposed to enhance the receptive field of the convolutional kernel. Furthermore, this paper presents a dataset of vehicle behavior. The advantages of the proposed method are verified by ablation experiments and comparative experiments while the real-time characteristics are maintained.