Abstract:
3D human behavior is a highly nonlinear spatiotemporal interaction process. Therefore, early behavior prediction is a challenging task, especially prediction with low obs...Show MoreMetadata
Abstract:
3D human behavior is a highly nonlinear spatiotemporal interaction process. Therefore, early behavior prediction is a challenging task, especially prediction with low observation rates in unsupervised mode. To this end, we propose a novel self-supervised early 3D behavior prediction frame-work that learns graph structures on hyperbolic manifold. Firstly, we employ the sequence construction of multi-dynamic key information to enlarge the key details of spatio-temporal behavior sequences, addressing the high redundancy between frames of spatio-temporal interaction. Secondly, for capturing dependencies among long-distance joints, we explore a unique graph Laplacian on hyperbolic manifold to perceive the subtle local difference within frames. Finally, we leverage the learned spatio-temporal features under different observation rates for progressive contrast, forming self-supervised signals. This facilitates the extraction of more discriminative global and local spatio-temporal information from early behavior sequences in unsupervised mode. Extensive experiments on three behavior datasets have demonstrated the superiority of our approach at low to medium observation rates.
Published in: IEEE Transactions on Cognitive and Developmental Systems ( Early Access )
Keywords assist with retrieval of results and provide a means to discovering other relevant content. Learn more.
- IEEE Keywords
- Index Terms
- Human Behavior ,
- Complete Sequences ,
- Unsupervised Learning ,
- Global Information ,
- Early Prediction ,
- Euclidean Space ,
- Spatiotemporal Characteristics ,
- Laplacian Matrix ,
- Graph Convolutional Network ,
- Graph Neural Networks ,
- Self-supervised Learning ,
- 3D Sequence ,
- Behavioral Sequences ,
- Graph Convolution ,
- Tangent Space ,
- 3D Prediction ,
- Temporal Convolutional Network ,
- Hyperbolic Space ,
- Spatiotemporal Sequence ,
- Spectral Convolution ,
- Graph Structure ,
- Sequence Changes ,
- Physical Concepts ,
- Action Recognition ,
- Graph Laplacian ,
- Predictor Of Behavior ,
- Momentum Factor ,
- Feature Representation ,
- Spatiotemporal Interactions ,
- Joint Point
- Author Keywords
Keywords assist with retrieval of results and provide a means to discovering other relevant content. Learn more.
- IEEE Keywords
- Index Terms
- Human Behavior ,
- Complete Sequences ,
- Unsupervised Learning ,
- Global Information ,
- Early Prediction ,
- Euclidean Space ,
- Spatiotemporal Characteristics ,
- Laplacian Matrix ,
- Graph Convolutional Network ,
- Graph Neural Networks ,
- Self-supervised Learning ,
- 3D Sequence ,
- Behavioral Sequences ,
- Graph Convolution ,
- Tangent Space ,
- 3D Prediction ,
- Temporal Convolutional Network ,
- Hyperbolic Space ,
- Spatiotemporal Sequence ,
- Spectral Convolution ,
- Graph Structure ,
- Sequence Changes ,
- Physical Concepts ,
- Action Recognition ,
- Graph Laplacian ,
- Predictor Of Behavior ,
- Momentum Factor ,
- Feature Representation ,
- Spatiotemporal Interactions ,
- Joint Point
- Author Keywords