Human action recognition with hidden Markov models and neural network derived poses

A human action recognition method is introduced that detects a set of actions in videos by a temporal expansion with hidden Markov models of a pose detection with an artificial neural network. The method was set-up and tested using eleven actions from the MOCAP motion capture database comprising 3,947 frames. A poses alphabet of fourteen relevant poses was defined to be learned by an artificial neural network. It was trained with the skeletons and a manual pose classification of 370 key frame images from the motions resulting in an accuracy of 83.5 %. Three actions prevalent in the used motions were chosen to be recognized with the interplay of one separate hidden Markov model for each action. From the output of the trained artificial neural network for all 1,891 frames of seven of the eleven motions together with a manual pose and action classification the matrices of the hidden Markov models for the four actions were calculated. A tailored maximum likelihood estimation based on the Viterbi algorithm generated an action proposal for each frame. The resulting overall accuracy was 83.2 % precision and 83.7 % recall for recognition of the actions.