Self-Supervised Hyperbolic Spectro-Temporal Graph Convolution Network for Early 3D Behavior Prediction | IEEE Journals & Magazine | IEEE Xplore

Self-Supervised Hyperbolic Spectro-Temporal Graph Convolution Network for Early 3D Behavior Prediction


Abstract:

3D human behavior is a highly nonlinear spatiotemporal interaction process. Therefore, early behavior prediction is a challenging task, especially prediction with low obs...Show More

Abstract:

3D human behavior is a highly nonlinear spatiotemporal interaction process. Therefore, early behavior prediction is a challenging task, especially prediction with low observation rates in unsupervised mode. To this end, we propose a novel self-supervised early 3D behavior prediction frame-work that learns graph structures on hyperbolic manifold. Firstly, we employ the sequence construction of multi-dynamic key information to enlarge the key details of spatio-temporal behavior sequences, addressing the high redundancy between frames of spatio-temporal interaction. Secondly, for capturing dependencies among long-distance joints, we explore a unique graph Laplacian on hyperbolic manifold to perceive the subtle local difference within frames. Finally, we leverage the learned spatio-temporal features under different observation rates for progressive contrast, forming self-supervised signals. This facilitates the extraction of more discriminative global and local spatio-temporal information from early behavior sequences in unsupervised mode. Extensive experiments on three behavior datasets have demonstrated the superiority of our approach at low to medium observation rates.
Page(s): 1 - 15
Date of Publication: 16 April 2025

ISSN Information:


Contact IEEE to Subscribe