TAL3×3: A Dataset and Algorithm for Temporal Action Localization in 3×3 Basketball

In the field of comprehensive video understanding, Temporal Action Localization (TAL) plays a vital role by precisely identifying when actions begin and end in untrimmed video sequences, enabling more accurate analysis of complex temporal dynamics. Despite their importance, existing datasets and algorithms in the sports field face significant challenges. Current datasets independently consider single-person or two-person event categories, overlooking the simultaneous occurrence of multiple actions and interactions in real-world environments. Existing studies primarily concentrate on mainstream sports like soccer, traditional basketball, and volleyball, whereas numerous sporting disciplines such as $3\!\times \!3$ basketball remain underserved with respect to specialized datasets and custom analytical frameworks. To address this issue, we propose a new real-world $3 \times 3$ basketball TAL (TAL $3 \times 3$ ) dataset and algorithm: TAL $3 \times 3$ dataset includes 3 single-person action classes (such as shooting a basketball) and 5 two-person interaction classes (such as passing the ball between players) with 633k human bounding boxes and 99572 action instances on 106k frames. To benchmark TAL $3 \times 3$ , we develop TAL $3 \times 3$ algorithm consisting of two distinct phases: 1) generation of action proposals, and 2) construction of representations for each proposal, followed by classification into specific interaction categories or background. Extensive experiments demonstrate that our method achieves remarkable performance with 60.09% mAP and 78.93% accuracy on our dataset, substantially outperforming existing approaches and establishing new benchmarks for complexity-aware action localization in team sports. We expect the TAL $3 \times 3$ will contribute to temporal action localization and basketball game analytics, while advancing the development of temporal contextual modeling techniques in the field of TAL. The dataset is available at https://github.com/open-starlab/TAL3×3