DHF-SLR: Dual-Hand Multi-Stream Fusion Network for Skeleton-Based Sign Language Recognition

The capacity of skeleton-based sign recognition to adapt well to complicated backgrounds and dynamic settings has drawn increasing interest. However, this method often suffers from incomplete hand skeleton information which leads to a low recognition accuracy. Multi-stream fusion is also often used in sign language recognition, but the method often uses manual weight assignment, which has problems such as subjectivity and inefficiency. This research proposes a two-handed multi-stream fusion network for sign language recognition based on the aforementioned issues, which extracts the skeleton information into three input streams, namely upper body, left hand and right hand, respectively, and inputs them into the network, which retains the completeness of the skeleton information to the maximum extent. Meanwhile, Bayesian optimization is introduced in the multi-stream fusion part to automatically assign weights to various prediction knots. Finally, with an accuracy of $95.93 \%$, the DHF-SLR framework was tested using the AUTSL sign language recognition dataset. which exceeds the current state-of-the-art methods.