Video based Gesture Recognition Method using Two-Stream 3D-ConvNet Network for Coal Mine Emergency Rescue Robot

The complexity and danger associated with underground coal mining operations constantly threats the miners' safety. Remote-controlled robot with man-in-loop can assist the emergency rescue. Thus, human gesture recognition is one of the most important issues. This paper proposes a method for recognizing emergency rescue gestures based on video understanding. First, the routine emergency rescue gestures of coal miners are decomposed, and a video dataset of underground emergency rescue gestures containing 9 gestures are constructed. Then a method based on the Two-Stream 3D-ConvNet network model is designed to complete gesture recognition. Feature extraction in the model adopts the method of calculating video optical flow information. The comparative experiments show that the model improves the recognition accuracy compared with 3D ConvNet and other methods on the constructed dataset, and verifies that the model has a better classification ability for the self-collected video dataset of mine emergency rescue gestures.