Loading web-font TeX/Main/Regular
S--Net:Semantic and Saliency Attention Network for Person Re-Identification | IEEE Journals & Magazine | IEEE Xplore

S^{2}-Net:Semantic and Saliency Attention Network for Person Re-Identification


Abstract:

Person re-identification is still a challenging task when moving objects or another person occludes the probe person. Mainstream methods based on even partitioning apply ...Show More

Abstract:

Person re-identification is still a challenging task when moving objects or another person occludes the probe person. Mainstream methods based on even partitioning apply an off-the-shelf human semantic parsing to highlight the non-collusion part. In this paper, we apply an attention branch to learn the human semantic partition to avoid misalignment introduced by even partitioning. In detail, we propose a semantic attention branch to learn 5 human semantic maps. We also note that some accessories or belongings, such as a hat, bag, may provide more informative clues to improve the person Re-ID. Human semantic parsing, however, usually treats non-human parts as distractions and discards them. To fetch the missing clues, we design a branch to capture the salient non-human parts. Finally, we merge the semantic and saliency attention to build an end-to-end network, named as S^{2}-Net. Specifically, to further improve Re-ID, we develop a trade-off weighting scheme between semantic and saliency attention and set the right weight with the actual scene. The extensive experiments show that S^{2}-Net gets the competitive performance. S^{2}-Net achieves 87.4% mAP on Market1501 and obtains 79.3%/56.1% rank-1/mAP on MSMT17 without semantic supervision. The source codes are available at https://github.com/upgirlnana/S2Net.
Published in: IEEE Transactions on Multimedia ( Volume: 25)
Page(s): 4387 - 4399
Date of Publication: 12 May 2022

ISSN Information:

Funding Agency:

Author image of Xuena Ren
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China
School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China
Xuena Ren received the M.S. degree in computer Science from the University of Science and Technology Beijing, Beijing, China. She is working toward the Ph.D. degree in cyberspace security with the Institute of Information Engineering, Chinese Academy of Sciences, School of Cyber Security, University of Chinese Academy Sciences, Beijing, China. Her research interests include Person re-identification, deep learning and comp...Show More
Xuena Ren received the M.S. degree in computer Science from the University of Science and Technology Beijing, Beijing, China. She is working toward the Ph.D. degree in cyberspace security with the Institute of Information Engineering, Chinese Academy of Sciences, School of Cyber Security, University of Chinese Academy Sciences, Beijing, China. Her research interests include Person re-identification, deep learning and comp...View more
Author image of Dongming Zhang
National Computer Network Emergency Response Technical Team Coordination Center of China, Beijing, China
Dongming Zhang (Member, IEEE) received the Ph.D. degree from the Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China, in 2006. In September 2006, he joined the Institute of Computing Technology, Chinese Academy of Sciences, and in March 2017, he joined the National Computer Network Emergency Response Technical Team/Coordination Center of China (CNCERT). He is currently a Professor with CNCERT. H...Show More
Dongming Zhang (Member, IEEE) received the Ph.D. degree from the Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China, in 2006. In September 2006, he joined the Institute of Computing Technology, Chinese Academy of Sciences, and in March 2017, he joined the National Computer Network Emergency Response Technical Team/Coordination Center of China (CNCERT). He is currently a Professor with CNCERT. H...View more
Author image of Xiuguo Bao
National Computer Network Emergency Response Technical Team Coordination Center of China, Beijing, China
Xiuguo Bao received the Ph.D. degree from the Institute of Computing Technology, Harbin Institute of Technology, Harbin, China, in 2005. He joined the National Computer Network Emergency Response Technical Team/Coordination Center of China (CNCERT) in 2005. He is currently a Professor with CNCERT. His research interests include informatin security and multimedia retrieval.
Xiuguo Bao received the Ph.D. degree from the Institute of Computing Technology, Harbin Institute of Technology, Harbin, China, in 2005. He joined the National Computer Network Emergency Response Technical Team/Coordination Center of China (CNCERT) in 2005. He is currently a Professor with CNCERT. His research interests include informatin security and multimedia retrieval.View more
Author image of Yongdong Zhang
School of Information Science and Technology, University of Science and Technology of China, Hefei, Anhui, China
Yongdong Zhang (Senior Member, IEEE) received the Ph.D. degree in electronic engineering from Tianjin University, Tianjin, China, in 2002. He is currently a Professor with the University of Science and Technology of China, Hefei, China. He has authored more than 100 refereed journals and conference papers. His research interests include multimedia content analysis and understanding, multimedia content security, video enco...Show More
Yongdong Zhang (Senior Member, IEEE) received the Ph.D. degree in electronic engineering from Tianjin University, Tianjin, China, in 2002. He is currently a Professor with the University of Science and Technology of China, Hefei, China. He has authored more than 100 refereed journals and conference papers. His research interests include multimedia content analysis and understanding, multimedia content security, video enco...View more

Author image of Xuena Ren
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China
School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China
Xuena Ren received the M.S. degree in computer Science from the University of Science and Technology Beijing, Beijing, China. She is working toward the Ph.D. degree in cyberspace security with the Institute of Information Engineering, Chinese Academy of Sciences, School of Cyber Security, University of Chinese Academy Sciences, Beijing, China. Her research interests include Person re-identification, deep learning and computer vision.
Xuena Ren received the M.S. degree in computer Science from the University of Science and Technology Beijing, Beijing, China. She is working toward the Ph.D. degree in cyberspace security with the Institute of Information Engineering, Chinese Academy of Sciences, School of Cyber Security, University of Chinese Academy Sciences, Beijing, China. Her research interests include Person re-identification, deep learning and computer vision.View more
Author image of Dongming Zhang
National Computer Network Emergency Response Technical Team Coordination Center of China, Beijing, China
Dongming Zhang (Member, IEEE) received the Ph.D. degree from the Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China, in 2006. In September 2006, he joined the Institute of Computing Technology, Chinese Academy of Sciences, and in March 2017, he joined the National Computer Network Emergency Response Technical Team/Coordination Center of China (CNCERT). He is currently a Professor with CNCERT. His research interests include video coding, multimedia content analysis, and understanding. Dr. Zhang is an ACM Member and Member of the China Computer Society.
Dongming Zhang (Member, IEEE) received the Ph.D. degree from the Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China, in 2006. In September 2006, he joined the Institute of Computing Technology, Chinese Academy of Sciences, and in March 2017, he joined the National Computer Network Emergency Response Technical Team/Coordination Center of China (CNCERT). He is currently a Professor with CNCERT. His research interests include video coding, multimedia content analysis, and understanding. Dr. Zhang is an ACM Member and Member of the China Computer Society.View more
Author image of Xiuguo Bao
National Computer Network Emergency Response Technical Team Coordination Center of China, Beijing, China
Xiuguo Bao received the Ph.D. degree from the Institute of Computing Technology, Harbin Institute of Technology, Harbin, China, in 2005. He joined the National Computer Network Emergency Response Technical Team/Coordination Center of China (CNCERT) in 2005. He is currently a Professor with CNCERT. His research interests include informatin security and multimedia retrieval.
Xiuguo Bao received the Ph.D. degree from the Institute of Computing Technology, Harbin Institute of Technology, Harbin, China, in 2005. He joined the National Computer Network Emergency Response Technical Team/Coordination Center of China (CNCERT) in 2005. He is currently a Professor with CNCERT. His research interests include informatin security and multimedia retrieval.View more
Author image of Yongdong Zhang
School of Information Science and Technology, University of Science and Technology of China, Hefei, Anhui, China
Yongdong Zhang (Senior Member, IEEE) received the Ph.D. degree in electronic engineering from Tianjin University, Tianjin, China, in 2002. He is currently a Professor with the University of Science and Technology of China, Hefei, China. He has authored more than 100 refereed journals and conference papers. His research interests include multimedia content analysis and understanding, multimedia content security, video encoding, and streaming media technology. He is an Editorial Board Member of Multimedia Systems journal and Neurocomputing. He was the recipient of the Best Paper Award in PCM 2013, ICIMCS 2013, and ICME 2010, and the Best Paper Candidate in ICME 2011.
Yongdong Zhang (Senior Member, IEEE) received the Ph.D. degree in electronic engineering from Tianjin University, Tianjin, China, in 2002. He is currently a Professor with the University of Science and Technology of China, Hefei, China. He has authored more than 100 refereed journals and conference papers. His research interests include multimedia content analysis and understanding, multimedia content security, video encoding, and streaming media technology. He is an Editorial Board Member of Multimedia Systems journal and Neurocomputing. He was the recipient of the Best Paper Award in PCM 2013, ICIMCS 2013, and ICME 2010, and the Best Paper Candidate in ICME 2011.View more

Contact IEEE to Subscribe

References

References is not available for this document.