Integrating Key-point Information for Sign Language Translation

Research in gloss-free sign language translation (GFSLT) has advanced significantly, but using key-point information remains underexplored. This paper introduces VK-SLT, a simple yet efficient model that integrates key-point data and video representation using a channel attention mechanism to enhance hand representation in feature extraction. Inspired by GFSLT-VLP, we employ CLIP pre-training to align visual and textual features. VK-SLT suppresses background noise, accelerates convergence, and improves performance. Preliminary evaluations show VK-SLT achieves better BLEU scores than GFSLT-VLP on a subset of the How to Sign dataset.