Attention-Driven ViT-CNN Ensemble for Robust Exercise Recognition Under Occlusion and Form Ambiguity

For the purpose of injury prevention fitness evaluation, accurate exercise classification is essential. However, singlebranch models are challenged by redundant motion patterns and form variations. To overcome these drawbacks, this paper suggests a hybrid deep learning strategy that combines a self-attention CNN with a Vision Transformer (ViT). The group uses temporal attention and spatial filtering to differentiate between visually similar exercises, such as pull-ups and jumping jacks. It uses velocity-based feature enhancement, selective data balancing, and sliding-window segmentation to manage occlusion and form variation. When tested on the Exercise Recognition Time Series dataset, the model outperforms ViT and CNN alone, achieving 9 4% accuracy and a macro-F¹ of 0.93. It demonstrates strong generalisation with a macro-F¹ of 0.97 and 97% accuracy on the publicly available Kaggle “Exercise Detection” dataset. The model works accurately.