Abstract:
Video person retrieval aims at matching video clips of the same person across non-overlapping camera views, where video sequences contain more comprehensive information, ...Show MoreMetadata
Abstract:
Video person retrieval aims at matching video clips of the same person across non-overlapping camera views, where video sequences contain more comprehensive information, e.g., temporal cues. How to extract useful temporal cues is the key to the success of a video person retrieval system. Gait, as a unique biometric modality indicating the way people walk, contains informative temporal information. To date, it is not clear how to fully utilize gait to boost the performance of video person retrieval. In this paper, to validate whether gait could help retrieve person in videos, we build a two-stream architecture, named appearance-gait network (AGNet), to jointly learn the appearance features and gait features from RGB video clips and silhouette video clips. We further explore how to fully utilize gait features to enhance the video feature representation. Specifically, we propose an appearance-gait attention module (AGA) to fuse a discriminative feature representation for the person retrieval task. Furthermore, to eliminate the requirement of silhouette video clips during inference, we propose a simple yet effective appearance-gait distillation module (AGD) which transfers the gait knowledge to appearance stream. As such, we are able to perform the enhanced video person retrieval without silhouette video clips, which makes the inference more flexible and practical. To the best of our knowledge, our work is the first to successfully introduce such appearance-gait knowledge distillation design for video person retrieval. We verify the effectiveness of the proposed methods on two large-scale challenging benchmarks of MARS and DukeMTMC-VideoReID. Extensive experiments demonstrate superior or comparable performance compared to the state-of-the-art methods while being much simpler. Source code is publicly available at https://github.com/yangyangkiki/Gait-Assisted-Video-Reid.
Published in: IEEE Transactions on Circuits and Systems for Video Technology ( Volume: 33, Issue: 2, February 2023)
Australian Institute for Machine Learning, The University of Adelaide, Adelaide, SA, Australia
Yang Zhao received the B.Sc. degree in computer science and technology from the Wuhan University of Technology, China, and the Ph.D. degree in artificial intelligence jointly from Griffith University and The University of Adelaide, Australia. She is currently a Research Fellow with the Australian Institute for Machine Learning (AIML), The University of Adelaide. She has been publishing peer-reviewed papers in journals and...Show More
Yang Zhao received the B.Sc. degree in computer science and technology from the Wuhan University of Technology, China, and the Ph.D. degree in artificial intelligence jointly from Griffith University and The University of Adelaide, Australia. She is currently a Research Fellow with the Australian Institute for Machine Learning (AIML), The University of Adelaide. She has been publishing peer-reviewed papers in journals and...View more
Australian Institute for Machine Learning, The University of Adelaide, Adelaide, SA, Australia
Xinlong Wang received the B.E. degree in automation from Tongji University, China. He is currently pursuing the Ph.D. degree with the School of Computer Science, The University of Adelaide, Australia. He has been publishing peer-reviewed papers in journals and conferences, including NeurIPS, CVPR, ECCV, and AAAI Conference on Artificial Intelligence. His research interests include computer vision and machine learning.
Xinlong Wang received the B.E. degree in automation from Tongji University, China. He is currently pursuing the Ph.D. degree with the School of Computer Science, The University of Adelaide, Australia. He has been publishing peer-reviewed papers in journals and conferences, including NeurIPS, CVPR, ECCV, and AAAI Conference on Artificial Intelligence. His research interests include computer vision and machine learning.View more
Institute for Integrated and Intelligent Systems, Griffith University, Nathan, QLD, Australia
Xiaohan Yu received the B.Sc. degree in computer science and technology from the Wuhan University of Technology, Wuhan, China, and the Ph.D. degree in engineering and built environment from Griffith University, Brisbane, QLD, Australia. He is currently a Research Fellow with the Institute for Integrated and Intelligent Systems, Griffith University. His main research interests include ultra-fine-grained visual categorizati...Show More
Xiaohan Yu received the B.Sc. degree in computer science and technology from the Wuhan University of Technology, Wuhan, China, and the Ph.D. degree in engineering and built environment from Griffith University, Brisbane, QLD, Australia. He is currently a Research Fellow with the Institute for Integrated and Intelligent Systems, Griffith University. His main research interests include ultra-fine-grained visual categorizati...View more
Australian Institute for Machine Learning, The University of Adelaide, Adelaide, SA, Australia
Chunlei Liu received the B.S. degree in information engineering from the Nanjing University of Aeronautics and Astronautics, China, in 2016, and the Ph.D. degree from the Department of Electrical and Information Engineering, Beihang University, China, in 2022. She visited the University of Adelaide with Prof. Chunhua Shen from 2019 to 2021. She is currently doing post-doctoral research with the Children’s Medical Research...Show More
Chunlei Liu received the B.S. degree in information engineering from the Nanjing University of Aeronautics and Astronautics, China, in 2016, and the Ph.D. degree from the Department of Electrical and Information Engineering, Beihang University, China, in 2022. She visited the University of Adelaide with Prof. Chunhua Shen from 2019 to 2021. She is currently doing post-doctoral research with the Children’s Medical Research...View more
Institute for Integrated and Intelligent Systems, Griffith University, Nathan, QLD, Australia
Yongsheng Gao (Senior Member, IEEE) received the B.Sc. and M.Sc. degrees in electronic engineering from Zhejiang University, Hangzhou, China, in 1985 and 1988, respectively, and the Ph.D. degree in computer engineering from Nanyang Technological University, Singapore. He is currently a Professor with the School of Engineering and Built Environment, Griffith University, and the Director of ARC Research Hub for Driving Farm...Show More
Yongsheng Gao (Senior Member, IEEE) received the B.Sc. and M.Sc. degrees in electronic engineering from Zhejiang University, Hangzhou, China, in 1985 and 1988, respectively, and the Ph.D. degree in computer engineering from Nanyang Technological University, Singapore. He is currently a Professor with the School of Engineering and Built Environment, Griffith University, and the Director of ARC Research Hub for Driving Farm...View more
Contents Australian Institute for Machine Learning, The University of Adelaide, Adelaide, SA, Australia
Yang Zhao received the B.Sc. degree in computer science and technology from the Wuhan University of Technology, China, and the Ph.D. degree in artificial intelligence jointly from Griffith University and The University of Adelaide, Australia. She is currently a Research Fellow with the Australian Institute for Machine Learning (AIML), The University of Adelaide. She has been publishing peer-reviewed papers in journals and conferences, including the IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Transactions on Circuits and Systems for Video Technology, Pattern Recognition, ICCV, and AAAI Conference on Artificial Intelligence. Her current research interests include computer vision and machine learning.
Yang Zhao received the B.Sc. degree in computer science and technology from the Wuhan University of Technology, China, and the Ph.D. degree in artificial intelligence jointly from Griffith University and The University of Adelaide, Australia. She is currently a Research Fellow with the Australian Institute for Machine Learning (AIML), The University of Adelaide. She has been publishing peer-reviewed papers in journals and conferences, including the IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Transactions on Circuits and Systems for Video Technology, Pattern Recognition, ICCV, and AAAI Conference on Artificial Intelligence. Her current research interests include computer vision and machine learning.View more
Australian Institute for Machine Learning, The University of Adelaide, Adelaide, SA, Australia
Xinlong Wang received the B.E. degree in automation from Tongji University, China. He is currently pursuing the Ph.D. degree with the School of Computer Science, The University of Adelaide, Australia. He has been publishing peer-reviewed papers in journals and conferences, including NeurIPS, CVPR, ECCV, and AAAI Conference on Artificial Intelligence. His research interests include computer vision and machine learning.
Xinlong Wang received the B.E. degree in automation from Tongji University, China. He is currently pursuing the Ph.D. degree with the School of Computer Science, The University of Adelaide, Australia. He has been publishing peer-reviewed papers in journals and conferences, including NeurIPS, CVPR, ECCV, and AAAI Conference on Artificial Intelligence. His research interests include computer vision and machine learning.View more
Institute for Integrated and Intelligent Systems, Griffith University, Nathan, QLD, Australia
Xiaohan Yu received the B.Sc. degree in computer science and technology from the Wuhan University of Technology, Wuhan, China, and the Ph.D. degree in engineering and built environment from Griffith University, Brisbane, QLD, Australia. He is currently a Research Fellow with the Institute for Integrated and Intelligent Systems, Griffith University. His main research interests include ultra-fine-grained visual categorization, computer vision, and pattern recognition.
Xiaohan Yu received the B.Sc. degree in computer science and technology from the Wuhan University of Technology, Wuhan, China, and the Ph.D. degree in engineering and built environment from Griffith University, Brisbane, QLD, Australia. He is currently a Research Fellow with the Institute for Integrated and Intelligent Systems, Griffith University. His main research interests include ultra-fine-grained visual categorization, computer vision, and pattern recognition.View more
Australian Institute for Machine Learning, The University of Adelaide, Adelaide, SA, Australia
Chunlei Liu received the B.S. degree in information engineering from the Nanjing University of Aeronautics and Astronautics, China, in 2016, and the Ph.D. degree from the Department of Electrical and Information Engineering, Beihang University, China, in 2022. She visited the University of Adelaide with Prof. Chunhua Shen from 2019 to 2021. She is currently doing post-doctoral research with the Children’s Medical Research Institute, Australia. Her research interests include deep learning and model binarization.
Chunlei Liu received the B.S. degree in information engineering from the Nanjing University of Aeronautics and Astronautics, China, in 2016, and the Ph.D. degree from the Department of Electrical and Information Engineering, Beihang University, China, in 2022. She visited the University of Adelaide with Prof. Chunhua Shen from 2019 to 2021. She is currently doing post-doctoral research with the Children’s Medical Research Institute, Australia. Her research interests include deep learning and model binarization.View more
Institute for Integrated and Intelligent Systems, Griffith University, Nathan, QLD, Australia
Yongsheng Gao (Senior Member, IEEE) received the B.Sc. and M.Sc. degrees in electronic engineering from Zhejiang University, Hangzhou, China, in 1985 and 1988, respectively, and the Ph.D. degree in computer engineering from Nanyang Technological University, Singapore. He is currently a Professor with the School of Engineering and Built Environment, Griffith University, and the Director of ARC Research Hub for Driving Farming Productivity and Disease Prevention, Australia. He had been the Leader of Biosecurity Group, Queensland Research Laboratory, National ICT Australia (ARC Centre of Excellence), a consultant of Panasonic Singapore Laboratories, and an Assistant Professor with the School of Computer Engineering, Nanyang Technological University. His research interests include smart farming, machine vision for agriculture, biosecurity, face recognition, biometrics, image retrieval, computer vision, pattern recognition, environmental informatics, and medical imaging.
Yongsheng Gao (Senior Member, IEEE) received the B.Sc. and M.Sc. degrees in electronic engineering from Zhejiang University, Hangzhou, China, in 1985 and 1988, respectively, and the Ph.D. degree in computer engineering from Nanyang Technological University, Singapore. He is currently a Professor with the School of Engineering and Built Environment, Griffith University, and the Director of ARC Research Hub for Driving Farming Productivity and Disease Prevention, Australia. He had been the Leader of Biosecurity Group, Queensland Research Laboratory, National ICT Australia (ARC Centre of Excellence), a consultant of Panasonic Singapore Laboratories, and an Assistant Professor with the School of Computer Engineering, Nanyang Technological University. His research interests include smart farming, machine vision for agriculture, biosecurity, face recognition, biometrics, image retrieval, computer vision, pattern recognition, environmental informatics, and medical imaging.View more
