Abstract:
Considering that the human brain always follows a coarse-to-fine (low-to-high spatial frequency) visual processing and fusion mechanism, we propose a coarse-to-fine feedb...Show MoreMetadata
Abstract:
Considering that the human brain always follows a coarse-to-fine (low-to-high spatial frequency) visual processing and fusion mechanism, we propose a coarse-to-fine feedback guidance based stereo image quality assessment (SIQA) network which considers a coarse-to-fine feedback guidance and adaptive dominant eye mechanism. The proposed network consists of two main sub-network streams, each of which has three branches to extract low, middle and high spatial frequency information in parallel. To better realize the guidance of the high-level features in the low spatial frequency branch to the low-level features in the high spatial frequency branch, an information feedback guidance module (IFGM) is proposed, which realizes a top-down guidance mechanism in each sub-network stream. Simultaneously, according to the theory of ocular dominance in human visual system (HVS), we design an adaptive bi-directional parallax-based binocular fusion module (BPBFM), which synthesizes two types of fusion feature by taking the left and right view features as dominant eye input. Furthermore, in order to obtain the better perceptual quality of stereo images, we design a weighted fusion strategy to weigh the quality scores from the two types of fusion features obtained by using an ensemble model with two multi-layer perceptrons (MLPs). The experimental results on four public stereo image datasets show that the proposed method is superior to the mainstream metrics and achieves an excellent performance.
Published in: IEEE Transactions on Multimedia ( Volume: 25)
Funding Agency:
School of Electrical and Information Engineering, Tianjin University, Tianjin, China
Yongli Chang received the master's degree in 2019 from the School of Electrical and Information Engineering, Tianjin University, Tianjin, China, where she has been working toward the Ph.D. degree since 2019. Her research interests include 2-D/3-D image quality evaluation, neural network, deep learning, and signal sparse representation.
Yongli Chang received the master's degree in 2019 from the School of Electrical and Information Engineering, Tianjin University, Tianjin, China, where she has been working toward the Ph.D. degree since 2019. Her research interests include 2-D/3-D image quality evaluation, neural network, deep learning, and signal sparse representation.View more
School of Electrical and Information Engineering, Tianjin University, Tianjin, China
Sumei Li (Member, IEEE) received the Ph.D. degree from Nankai University, Tianjin, China. Since 2006, she has been an Associate Professor with Communication Engineering Department, Tianjin University, Tianjin, China. She chaired a National 863 Project, a National Natural Science Foundation, and a Key Fund in Tianjin. Her research interests include 3-D image/video transmission, processing and quality evaluation, depth/imag...Show More
Sumei Li (Member, IEEE) received the Ph.D. degree from Nankai University, Tianjin, China. Since 2006, she has been an Associate Professor with Communication Engineering Department, Tianjin University, Tianjin, China. She chaired a National 863 Project, a National Natural Science Foundation, and a Key Fund in Tianjin. Her research interests include 3-D image/video transmission, processing and quality evaluation, depth/imag...View more
School of Electrical and Information Engineering, Tianjin University, Tianjin, China
Anqi Liu received the master's degree with the School of Electrical and Information Engineering, Tianjin University, Tianjin, China, in 2018. She is currently working toward the Ph.D. degree with the school of Electrical and Information Engineering, Tianjin University. Her research interests include single/stereo image super-resolution, neural network, and deep learning.
Anqi Liu received the master's degree with the School of Electrical and Information Engineering, Tianjin University, Tianjin, China, in 2018. She is currently working toward the Ph.D. degree with the school of Electrical and Information Engineering, Tianjin University. Her research interests include single/stereo image super-resolution, neural network, and deep learning.View more
School of Electrical and Information Engineering, Tianjin University, Tianjin, China
Jie Jin has been a Professor with the School of Electrical and Information Engineering, Tianjin University, since 2004. She has presided over the National Natural Science Foundation of China, Tianjin Natural Science Foundation and national major science and technology special sub projects. Her research interests include intelligent instrument and photoelectric technology, optical fiber communication and devices, behavior ...Show More
Jie Jin has been a Professor with the School of Electrical and Information Engineering, Tianjin University, since 2004. She has presided over the National Natural Science Foundation of China, Tianjin Natural Science Foundation and national major science and technology special sub projects. Her research interests include intelligent instrument and photoelectric technology, optical fiber communication and devices, behavior ...View more
School of Engineering and Mathematical Sciences, La Trobe University, Melbourne, VIC, Australia
Wei Xiang (Senior Member, IEEE) is currently a Cisco Research Chair of AI and IoT and the Director of the Cisco-La Trobe Centre for AI and IoT with La Trobe University, New Delhi, India. He was the Foundation Chair and the Head of Discipline of IoT Engineering with James Cook University, Cairns, Australia. His research interests include the Internet of Things, wireless communications, machine learning for IoT data analyti...Show More
Wei Xiang (Senior Member, IEEE) is currently a Cisco Research Chair of AI and IoT and the Director of the Cisco-La Trobe Centre for AI and IoT with La Trobe University, New Delhi, India. He was the Foundation Chair and the Head of Discipline of IoT Engineering with James Cook University, Cairns, Australia. His research interests include the Internet of Things, wireless communications, machine learning for IoT data analyti...View more
Contents School of Electrical and Information Engineering, Tianjin University, Tianjin, China
Yongli Chang received the master's degree in 2019 from the School of Electrical and Information Engineering, Tianjin University, Tianjin, China, where she has been working toward the Ph.D. degree since 2019. Her research interests include 2-D/3-D image quality evaluation, neural network, deep learning, and signal sparse representation.
Yongli Chang received the master's degree in 2019 from the School of Electrical and Information Engineering, Tianjin University, Tianjin, China, where she has been working toward the Ph.D. degree since 2019. Her research interests include 2-D/3-D image quality evaluation, neural network, deep learning, and signal sparse representation.View more
School of Electrical and Information Engineering, Tianjin University, Tianjin, China
Sumei Li (Member, IEEE) received the Ph.D. degree from Nankai University, Tianjin, China. Since 2006, she has been an Associate Professor with Communication Engineering Department, Tianjin University, Tianjin, China. She chaired a National 863 Project, a National Natural Science Foundation, and a Key Fund in Tianjin. Her research interests include 3-D image/video transmission, processing and quality evaluation, depth/image SR reconstruction, sparse representation, neural network, and deep learning. Dr. Li is a Member of China's Neural Network Committee.
Sumei Li (Member, IEEE) received the Ph.D. degree from Nankai University, Tianjin, China. Since 2006, she has been an Associate Professor with Communication Engineering Department, Tianjin University, Tianjin, China. She chaired a National 863 Project, a National Natural Science Foundation, and a Key Fund in Tianjin. Her research interests include 3-D image/video transmission, processing and quality evaluation, depth/image SR reconstruction, sparse representation, neural network, and deep learning. Dr. Li is a Member of China's Neural Network Committee.View more
School of Electrical and Information Engineering, Tianjin University, Tianjin, China
Anqi Liu received the master's degree with the School of Electrical and Information Engineering, Tianjin University, Tianjin, China, in 2018. She is currently working toward the Ph.D. degree with the school of Electrical and Information Engineering, Tianjin University. Her research interests include single/stereo image super-resolution, neural network, and deep learning.
Anqi Liu received the master's degree with the School of Electrical and Information Engineering, Tianjin University, Tianjin, China, in 2018. She is currently working toward the Ph.D. degree with the school of Electrical and Information Engineering, Tianjin University. Her research interests include single/stereo image super-resolution, neural network, and deep learning.View more
School of Electrical and Information Engineering, Tianjin University, Tianjin, China
Jie Jin has been a Professor with the School of Electrical and Information Engineering, Tianjin University, since 2004. She has presided over the National Natural Science Foundation of China, Tianjin Natural Science Foundation and national major science and technology special sub projects. Her research interests include intelligent instrument and photoelectric technology, optical fiber communication and devices, behavior detection based on neural network, image processing, and microwave antenna design of microwave devices.
Jie Jin has been a Professor with the School of Electrical and Information Engineering, Tianjin University, since 2004. She has presided over the National Natural Science Foundation of China, Tianjin Natural Science Foundation and national major science and technology special sub projects. Her research interests include intelligent instrument and photoelectric technology, optical fiber communication and devices, behavior detection based on neural network, image processing, and microwave antenna design of microwave devices.View more
School of Engineering and Mathematical Sciences, La Trobe University, Melbourne, VIC, Australia
Wei Xiang (Senior Member, IEEE) is currently a Cisco Research Chair of AI and IoT and the Director of the Cisco-La Trobe Centre for AI and IoT with La Trobe University, New Delhi, India. He was the Foundation Chair and the Head of Discipline of IoT Engineering with James Cook University, Cairns, Australia. His research interests include the Internet of Things, wireless communications, machine learning for IoT data analytics, and computer vision. He was the recipient of Four Best Paper Awards at WiSATS'2019, WCSP'2015, IEEE WCNC'2011, and ICWMC'2009. He was named a Queensland International Fellow during 2010–2011 by the Queensland Government of Australia, an Endeavour Research Fellow during 2012–2013 by the Commonwealth Government of Australia, a Smart Futures Fellow during 2012–2015 by the Queensland Government of Australia, and a JSPS Invitational Fellow jointly by the Australian Academy of Science and Japanese Society for Promotion of Science during 2014–2015. He was the Vice Chair of IEEE Northern Australia Section from 2016 to 2020. He is currently an Associate Editor for IEEE Communications Surveys & Tutorials, IEEE Internet of Things Journal, IEEE Access, and Nature journal of Scientific Reports.
Wei Xiang (Senior Member, IEEE) is currently a Cisco Research Chair of AI and IoT and the Director of the Cisco-La Trobe Centre for AI and IoT with La Trobe University, New Delhi, India. He was the Foundation Chair and the Head of Discipline of IoT Engineering with James Cook University, Cairns, Australia. His research interests include the Internet of Things, wireless communications, machine learning for IoT data analytics, and computer vision. He was the recipient of Four Best Paper Awards at WiSATS'2019, WCSP'2015, IEEE WCNC'2011, and ICWMC'2009. He was named a Queensland International Fellow during 2010–2011 by the Queensland Government of Australia, an Endeavour Research Fellow during 2012–2013 by the Commonwealth Government of Australia, a Smart Futures Fellow during 2012–2015 by the Queensland Government of Australia, and a JSPS Invitational Fellow jointly by the Australian Academy of Science and Japanese Society for Promotion of Science during 2014–2015. He was the Vice Chair of IEEE Northern Australia Section from 2016 to 2020. He is currently an Associate Editor for IEEE Communications Surveys & Tutorials, IEEE Internet of Things Journal, IEEE Access, and Nature journal of Scientific Reports.View more
