Two-person Interaction Recognition Algorithm Based on the Joint Points Attention Motion Features Map

The human interaction recognition based on joint point date and RGB video has broad application prospects. RGB video contains information such as the appearance and texture of the human body. The joint point data is the coordinate’s position of three-dimensional skeleton during the human body movement. The two features have good complementarity. However, the fusion methods currently studied mostly adopt the decision-level fusion method, leading to less change of the fusion form and failing to make a better use of the complementarity of the two data sources. In response to the above problems, a human interaction recognition algorithm is proposed by fusing joint point data and RGB video in the feature layer. The RGB video is used as the main identification information, and the joint point data is used as auxiliary identification information in the proposed method. First the joint points attention motion features map is constructed by analyzing the joint point data to obtain the main motion part, and paying attention to the corresponding position of the RGB motion feature. Then the VGG19 network is used to extract deep features of the joint points attention motion features map. Finally the Bi-LSTM is introduced to perform timing modeling to obtain the final recognition result. The method was tested on the international standard NTU RGB+D database. The test results show that the algorithm greatly improves the accuracy of similar actions recognition, which proves the feasibility of the algorithm. This method provides a novel solution based on the fusion of RGB video and joint point information. It is simple and easy to implement, and has good practicability.