Abstract:
For autonomous vehicles (AVs), visual perception techniques based on sensors like cameras play crucial roles in information acquisition and processing. In various compute...Show MoreMetadata
Abstract:
For autonomous vehicles (AVs), visual perception techniques based on sensors like cameras play crucial roles in information acquisition and processing. In various computer perception tasks for AVs, it may be helpful to match landmark patches taken by an onboard camera with other landmark patches captured at a different time or saved in a street scene image database. To perform matching under challenging driving environments caused by changing seasons, weather, and illumination, we utilize the spatial neighborhood information of each patch. We propose an approach, named RobustMat, which derives its robustness to perturbations from neural differential equations. A convolutional neural ODE diffusion module is used to learn the feature representation for the landmark patches. A graph neural PDE diffusion module then aggregates information from neighboring landmark patches in the street scene. Finally, feature similarity learning outputs the final matching score. Our approach is evaluated on several street scene datasets and demonstrated to achieve state-of-the-art matching results under environmental perturbations.
Published in: IEEE Transactions on Image Processing ( Volume: 32)
Funding Agency:
School of Electrical and Electronic Engineering, Nanyang Technological University, Jurong West, Singapore
Rui She received the B.S. degree in communication engineering from Jilin University, China, in 2015, and the Ph.D. degree in electronic engineering from Tsinghua University, China, in 2020. From December 2018 to June 2019, he was a Visiting Scholar with Columbia University, New York, NY, USA. He is currently a Research Fellow with Nanyang Technological University, Singapore. His research interests include intelligent sign...Show More
Rui She received the B.S. degree in communication engineering from Jilin University, China, in 2015, and the Ph.D. degree in electronic engineering from Tsinghua University, China, in 2020. From December 2018 to June 2019, he was a Visiting Scholar with Columbia University, New York, NY, USA. He is currently a Research Fellow with Nanyang Technological University, Singapore. His research interests include intelligent sign...View more
School of Electrical and Electronic Engineering, Nanyang Technological University, Jurong West, Singapore
Qiyu Kang received the B.S. degree in electronic information science and technology from the University of Science and Technology of China in 2015 and the Ph.D. degree from Nanyang Technological University (NTU), Singapore, in 2019. He is currently a Research Fellow with NTU. His research domains are focused on multi-sensor localization and mapping, robust machine learning, and graph neural networks.
Qiyu Kang received the B.S. degree in electronic information science and technology from the University of Science and Technology of China in 2015 and the Ph.D. degree from Nanyang Technological University (NTU), Singapore, in 2019. He is currently a Research Fellow with NTU. His research domains are focused on multi-sensor localization and mapping, robust machine learning, and graph neural networks.View more
School of Electrical and Electronic Engineering, Nanyang Technological University, Jurong West, Singapore
Sijie Wang received the B.Eng. degree from the Harbin Institute of Technology, China, in 2020, and the M.Sc. degree from Nanyang Technological University, Singapore, in 2021, where he is currently pursuing the Ph.D. degree with the School of Electrical and Electronic Engineering. His research interests include computer vision, autonomous driving, deep graph learning, and localization with multi-modal sensors.
Sijie Wang received the B.Eng. degree from the Harbin Institute of Technology, China, in 2020, and the M.Sc. degree from Nanyang Technological University, Singapore, in 2021, where he is currently pursuing the Ph.D. degree with the School of Electrical and Electronic Engineering. His research interests include computer vision, autonomous driving, deep graph learning, and localization with multi-modal sensors.View more
Continental-NTU Corporate Lab, Nanyang Technological University, Jurong West, Singapore
Yuán-Ruì Yáng received the B.Eng. degree from Nanyang Technological University, Singapore, in 2012, the master’s degree from The Australian National University in 2014, and the Ph.D. degree from the National University of Singapore in 2020. He is currently a Research Fellow with Nanyang Technological University. He has worked on research and development projects for both university and industry in robotics control, naviga...Show More
Yuán-Ruì Yáng received the B.Eng. degree from Nanyang Technological University, Singapore, in 2012, the master’s degree from The Australian National University in 2014, and the Ph.D. degree from the National University of Singapore in 2020. He is currently a Research Fellow with Nanyang Technological University. He has worked on research and development projects for both university and industry in robotics control, naviga...View more
School of Electrical and Electronic Engineering, Nanyang Technological University, Jurong West, Singapore
Kai Zhao received the B.Eng. degree in electrical engineering and automation from the Huazhong University of Science and Technology, China, in 2017, and the M.Sc. degree in electrical engineering from the National University of Singapore in 2019. He is currently pursuing the Ph.D. degree with the School of Electrical and Electronic Engineering, Nanyang Technological University. His research interests include graph learnin...Show More
Kai Zhao received the B.Eng. degree in electrical engineering and automation from the Huazhong University of Science and Technology, China, in 2017, and the M.Sc. degree in electrical engineering from the National University of Singapore in 2019. He is currently pursuing the Ph.D. degree with the School of Electrical and Electronic Engineering, Nanyang Technological University. His research interests include graph learnin...View more
School of Electrical and Electronic Engineering, Nanyang Technological University, Jurong West, Singapore
Yang Song received the B.Eng. degree in communication engineering from the Zhejiang University City College, Hangzhou, China, in 2007, and the M.Eng. and Ph.D. degrees in electronic and information engineering from The Hong Kong Polytechnic University, Hong Kong, in 2008 and 2013, respectively. He was a Research Associate with The Hong Kong Polytechnic University until 2014. From 2014 to 2016, he was a Postdoctoral Resear...Show More
Yang Song received the B.Eng. degree in communication engineering from the Zhejiang University City College, Hangzhou, China, in 2007, and the M.Eng. and Ph.D. degrees in electronic and information engineering from The Hong Kong Polytechnic University, Hong Kong, in 2008 and 2013, respectively. He was a Research Associate with The Hong Kong Polytechnic University until 2014. From 2014 to 2016, he was a Postdoctoral Resear...View more
School of Electrical and Electronic Engineering, Nanyang Technological University, Jurong West, Singapore
Wee Peng Tay (Senior Member, IEEE) received the B.S. degree in electrical engineering and mathematics, the M.S. degree in electrical engineering from Stanford University, Stanford, CA, USA, in 2002, and the Ph.D. degree in electrical engineering and computer science from the Massachusetts Institute of Technology, Cambridge, MA, USA, in 2008. He is currently a Professor of signal and information processing with the School ...Show More
Wee Peng Tay (Senior Member, IEEE) received the B.S. degree in electrical engineering and mathematics, the M.S. degree in electrical engineering from Stanford University, Stanford, CA, USA, in 2002, and the Ph.D. degree in electrical engineering and computer science from the Massachusetts Institute of Technology, Cambridge, MA, USA, in 2008. He is currently a Professor of signal and information processing with the School ...View more
Contents School of Electrical and Electronic Engineering, Nanyang Technological University, Jurong West, Singapore
Rui She received the B.S. degree in communication engineering from Jilin University, China, in 2015, and the Ph.D. degree in electronic engineering from Tsinghua University, China, in 2020. From December 2018 to June 2019, he was a Visiting Scholar with Columbia University, New York, NY, USA. He is currently a Research Fellow with Nanyang Technological University, Singapore. His research interests include intelligent signal and information processing, autonomous driving, 2D and 3D computer vision, deep learning, and the Artificial Intelligence of Things.
Rui She received the B.S. degree in communication engineering from Jilin University, China, in 2015, and the Ph.D. degree in electronic engineering from Tsinghua University, China, in 2020. From December 2018 to June 2019, he was a Visiting Scholar with Columbia University, New York, NY, USA. He is currently a Research Fellow with Nanyang Technological University, Singapore. His research interests include intelligent signal and information processing, autonomous driving, 2D and 3D computer vision, deep learning, and the Artificial Intelligence of Things.View more
School of Electrical and Electronic Engineering, Nanyang Technological University, Jurong West, Singapore
Qiyu Kang received the B.S. degree in electronic information science and technology from the University of Science and Technology of China in 2015 and the Ph.D. degree from Nanyang Technological University (NTU), Singapore, in 2019. He is currently a Research Fellow with NTU. His research domains are focused on multi-sensor localization and mapping, robust machine learning, and graph neural networks.
Qiyu Kang received the B.S. degree in electronic information science and technology from the University of Science and Technology of China in 2015 and the Ph.D. degree from Nanyang Technological University (NTU), Singapore, in 2019. He is currently a Research Fellow with NTU. His research domains are focused on multi-sensor localization and mapping, robust machine learning, and graph neural networks.View more
School of Electrical and Electronic Engineering, Nanyang Technological University, Jurong West, Singapore
Sijie Wang received the B.Eng. degree from the Harbin Institute of Technology, China, in 2020, and the M.Sc. degree from Nanyang Technological University, Singapore, in 2021, where he is currently pursuing the Ph.D. degree with the School of Electrical and Electronic Engineering. His research interests include computer vision, autonomous driving, deep graph learning, and localization with multi-modal sensors.
Sijie Wang received the B.Eng. degree from the Harbin Institute of Technology, China, in 2020, and the M.Sc. degree from Nanyang Technological University, Singapore, in 2021, where he is currently pursuing the Ph.D. degree with the School of Electrical and Electronic Engineering. His research interests include computer vision, autonomous driving, deep graph learning, and localization with multi-modal sensors.View more
Continental-NTU Corporate Lab, Nanyang Technological University, Jurong West, Singapore
Yuán-Ruì Yáng received the B.Eng. degree from Nanyang Technological University, Singapore, in 2012, the master’s degree from The Australian National University in 2014, and the Ph.D. degree from the National University of Singapore in 2020. He is currently a Research Fellow with Nanyang Technological University. He has worked on research and development projects for both university and industry in robotics control, navigation, computer vision, UAV, and UAM. His research interests include multi-robot navigation, coverage control, intelligent control, robotics, eVTOL aerial vehicles, and deep reinforcement learning.
Yuán-Ruì Yáng received the B.Eng. degree from Nanyang Technological University, Singapore, in 2012, the master’s degree from The Australian National University in 2014, and the Ph.D. degree from the National University of Singapore in 2020. He is currently a Research Fellow with Nanyang Technological University. He has worked on research and development projects for both university and industry in robotics control, navigation, computer vision, UAV, and UAM. His research interests include multi-robot navigation, coverage control, intelligent control, robotics, eVTOL aerial vehicles, and deep reinforcement learning.View more
School of Electrical and Electronic Engineering, Nanyang Technological University, Jurong West, Singapore
Kai Zhao received the B.Eng. degree in electrical engineering and automation from the Huazhong University of Science and Technology, China, in 2017, and the M.Sc. degree in electrical engineering from the National University of Singapore in 2019. He is currently pursuing the Ph.D. degree with the School of Electrical and Electronic Engineering, Nanyang Technological University. His research interests include graph learning and adversarial robustness in machine learning.
Kai Zhao received the B.Eng. degree in electrical engineering and automation from the Huazhong University of Science and Technology, China, in 2017, and the M.Sc. degree in electrical engineering from the National University of Singapore in 2019. He is currently pursuing the Ph.D. degree with the School of Electrical and Electronic Engineering, Nanyang Technological University. His research interests include graph learning and adversarial robustness in machine learning.View more
School of Electrical and Electronic Engineering, Nanyang Technological University, Jurong West, Singapore
Yang Song received the B.Eng. degree in communication engineering from the Zhejiang University City College, Hangzhou, China, in 2007, and the M.Eng. and Ph.D. degrees in electronic and information engineering from The Hong Kong Polytechnic University, Hong Kong, in 2008 and 2013, respectively. He was a Research Associate with The Hong Kong Polytechnic University until 2014. From 2014 to 2016, he was a Postdoctoral Research Associate with Universitä Paderborn, Paderborn, Germany. From 2016 to 2022, he was a Senior Research Fellow with Nanyang Technological University, Singapore. Since 2022, he has been a Senior Data Scientist with C3 AI. He is with the School of Electrical and Electronic Engineering, Nanyang Technological University. His current research interests include signal processing and adversarial machine learning. He is currently an Associate Editor of IET Signal Processing.
Yang Song received the B.Eng. degree in communication engineering from the Zhejiang University City College, Hangzhou, China, in 2007, and the M.Eng. and Ph.D. degrees in electronic and information engineering from The Hong Kong Polytechnic University, Hong Kong, in 2008 and 2013, respectively. He was a Research Associate with The Hong Kong Polytechnic University until 2014. From 2014 to 2016, he was a Postdoctoral Research Associate with Universitä Paderborn, Paderborn, Germany. From 2016 to 2022, he was a Senior Research Fellow with Nanyang Technological University, Singapore. Since 2022, he has been a Senior Data Scientist with C3 AI. He is with the School of Electrical and Electronic Engineering, Nanyang Technological University. His current research interests include signal processing and adversarial machine learning. He is currently an Associate Editor of IET Signal Processing.View more
School of Electrical and Electronic Engineering, Nanyang Technological University, Jurong West, Singapore
Wee Peng Tay (Senior Member, IEEE) received the B.S. degree in electrical engineering and mathematics, the M.S. degree in electrical engineering from Stanford University, Stanford, CA, USA, in 2002, and the Ph.D. degree in electrical engineering and computer science from the Massachusetts Institute of Technology, Cambridge, MA, USA, in 2008. He is currently a Professor of signal and information processing with the School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore. His research interests include signal and information processing over networks, distributed inference and estimation, statistical privacy, and robust machine learning. He received the Tan Chin Tuan Exchange Fellowship in 2015. He is the coauthor of the Best Student Paper Award from the Asilomar Conference on Signals, Systems, and Computers in 2012, the IEEE Signal Processing Society Young Author Best Paper Award in 2016, and the Best Paper Award from the International Conference on Smart Power and Internet Energy Systems in 2022. He was an Associate Editor of IEEE Transactions on Signal Processing (2015–2019). He is also an Associate Editor of IEEE Transactions on Signal and Information Processing Over Networks, an Editor of the IEEE Transactions on Wireless Communications, and an Editor of the IEEE Open Journal of Vehicular Technology.
Wee Peng Tay (Senior Member, IEEE) received the B.S. degree in electrical engineering and mathematics, the M.S. degree in electrical engineering from Stanford University, Stanford, CA, USA, in 2002, and the Ph.D. degree in electrical engineering and computer science from the Massachusetts Institute of Technology, Cambridge, MA, USA, in 2008. He is currently a Professor of signal and information processing with the School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore. His research interests include signal and information processing over networks, distributed inference and estimation, statistical privacy, and robust machine learning. He received the Tan Chin Tuan Exchange Fellowship in 2015. He is the coauthor of the Best Student Paper Award from the Asilomar Conference on Signals, Systems, and Computers in 2012, the IEEE Signal Processing Society Young Author Best Paper Award in 2016, and the Best Paper Award from the International Conference on Smart Power and Internet Energy Systems in 2022. He was an Associate Editor of IEEE Transactions on Signal Processing (2015–2019). He is also an Associate Editor of IEEE Transactions on Signal and Information Processing Over Networks, an Editor of the IEEE Transactions on Wireless Communications, and an Editor of the IEEE Open Journal of Vehicular Technology.View more
