A Sign Language Interactive System based on Multi-feature Fusion

The sign language interaction system not only helps improve the communication mode of deaf-mutes, but also helps us explore gesture interaction technology. In this paper, the MYO armband was used to collect 6-channel inertial signals and 8-channel sEMG signals, and 8 subjects were invited to establish 35 Chinese sign language data sets, and a complete sign language interactive system was developed with a mobile application. In the aspect of algorithm, a segmentation method based on the comprehensive judgment of average energy and volatility is proposed to extract the starting and ending points of active segments more accurately. In addition, a multi-feature fusion method is proposed in feature selection, and the accuracy of the best subset is 98.12% verified by SVM classifier. At the same time, it is determined that the classification effect is better when the feature dimension is about 30.