Loading [MathJax]/extensions/MathZoom.js
Object Detection Using ESRGAN With a Sequential Transfer Learning on Remote Sensing Embedded Systems | IEEE Journals & Magazine | IEEE Xplore

Object Detection Using ESRGAN With a Sequential Transfer Learning on Remote Sensing Embedded Systems


The field of remote sensing has experienced rapid advancement owing to the widespread utilization of image sensors, drones, and satellites for data collection. However, o...

Abstract:

The field of remote sensing has experienced rapid advancement owing to the widespread utilization of image sensors, drones, and satellites for data collection. However, o...Show More

Abstract:

The field of remote sensing has experienced rapid advancement owing to the widespread utilization of image sensors, drones, and satellites for data collection. However, object detection in remote sensing poses challenges owing to small objects with low resolution (LR), complex scenes, and limited data for model training. Conventional methods rely on computationally intensive models and hardware setups that are not suitable for real-time detection. To address this issue, we propose a novel sequential transfer learning method based on generative adversarial networks (GANs) that generate super-resolved data from LR for embedded systems, enabling improved performance with limited data by combining learning from both heterogeneous and homogeneous data. Additionally, we train the model sequentially, starting with the easiest data and progressing to the most complex based on the complexity levels determined by the GAN-generated images. The GAN model is trained on a diverse dataset of images and learned to generate high-resolution images from the LR, capturing finer object details for enhanced accuracy and localization capabilities. The proposed method acquires more robust features and enhances the generalizability and convergence of the model. Furthermore, the trained model of the proposed method is deployed on embedded platforms, such as Nvidia’s Jetson Nano and AGX Orin, for real-time remote-sensing object detection, with satisfactory detection performance. Evaluation metrics, such as mAP@0.5, mAP@0.5–0.95, and F1 score were used to assess the object detection accuracy. The experimental results demonstrated a significant improvement in accuracy when the proposed method was implemented with YOLOv7, achieving detection performance scores of 99.21, 98.57, 93.71, 78.38, 75.73, 48.68, 0.971, 0.971, and 0.911 on the VEDAI-VISIBLE, VEDAI-IR, and DOTA datasets, respectively.
The field of remote sensing has experienced rapid advancement owing to the widespread utilization of image sensors, drones, and satellites for data collection. However, o...
Published in: IEEE Access ( Volume: 12)
Page(s): 102313 - 102327
Date of Publication: 23 July 2024
Electronic ISSN: 2169-3536

Funding Agency:

Author image of Yogendra Rao Musunuri
Department of Control and Instrumentation Engineering, Changwon National University, Changwon, Republic of Korea
Yogendra Rao Musunuri received the B.Tech. degree in electronics and communication engineering from DPREC, Jawaharlal Nehru Technological University (JNTU), Hyderabad, India, in 2012, and the M.E. degree in image processing and computer vision from Busan University of Foreign Studies (BUFS), Busan, South Korea, in 2017. He is currently pursuing the Ph.D. degree in the control and instrumentation engineering with Changwon ...Show More
Yogendra Rao Musunuri received the B.Tech. degree in electronics and communication engineering from DPREC, Jawaharlal Nehru Technological University (JNTU), Hyderabad, India, in 2012, and the M.E. degree in image processing and computer vision from Busan University of Foreign Studies (BUFS), Busan, South Korea, in 2017. He is currently pursuing the Ph.D. degree in the control and instrumentation engineering with Changwon ...View more
Author image of Changwon Kim
School of Electrical, Electronics, and Control Engineering, Changwon National University, Changwon, Republic of Korea
Changwon Kim received the B.S., M.S., and Ph.D. degrees in the electrical and electronic engineering from Yonsei University, Republic of Korea, in 2002, 2004, and 2010, respectively. From 2010 to 2015, he was a Senior Researcher with Samsung Electronics, Suwon, South Korea. From 2015 to 2022, he was a Patent Examiner with Korean Intellectual Property Office. Since 2022, he has been with Changwon National University, where...Show More
Changwon Kim received the B.S., M.S., and Ph.D. degrees in the electrical and electronic engineering from Yonsei University, Republic of Korea, in 2002, 2004, and 2010, respectively. From 2010 to 2015, he was a Senior Researcher with Samsung Electronics, Suwon, South Korea. From 2015 to 2022, he was a Patent Examiner with Korean Intellectual Property Office. Since 2022, he has been with Changwon National University, where...View more
Author image of Oh-Seol Kwon
School of Electrical, Electronics, and Control Engineering, Changwon National University, Changwon, Republic of Korea
Oh-Seol Kwon received the B.S. and M.S. degrees in electrical engineering and computer science and the Ph.D. degree in electronics from Kyungpook National University, Republic of Korea, in 2002, 2004, and 2008, respectively. From 2008 to 2010, he was a Postdoctoral Research Fellow with New York University, New York, NY, USA. From 2010 to 2011, he was a Senior Researcher with the Visual Display Division, Samsung Electronic...Show More
Oh-Seol Kwon received the B.S. and M.S. degrees in electrical engineering and computer science and the Ph.D. degree in electronics from Kyungpook National University, Republic of Korea, in 2002, 2004, and 2008, respectively. From 2008 to 2010, he was a Postdoctoral Research Fellow with New York University, New York, NY, USA. From 2010 to 2011, he was a Senior Researcher with the Visual Display Division, Samsung Electronic...View more
Author image of Sun-Yuan Kung
School of Electrical and Computer Engineering, Princeton University, Princeton, NJ, USA
Sun-Yuan Kung (Life Fellow, IEEE) is currently a Professor with the Department of Electrical Engineering, Princeton University, Princeton, NJ, USA. He has authored or co-authored over 500 technical publications and numerous textbooks, including the VLSI Array Processors (Prentice-Hall, 1988), the Digital Neural Networks (Prentice-Hall, 1993), the Principal Component Neural Networks (Wiley, 1996), the Biometric Authenticat...Show More
Sun-Yuan Kung (Life Fellow, IEEE) is currently a Professor with the Department of Electrical Engineering, Princeton University, Princeton, NJ, USA. He has authored or co-authored over 500 technical publications and numerous textbooks, including the VLSI Array Processors (Prentice-Hall, 1988), the Digital Neural Networks (Prentice-Hall, 1993), the Principal Component Neural Networks (Wiley, 1996), the Biometric Authenticat...View more

Author image of Yogendra Rao Musunuri
Department of Control and Instrumentation Engineering, Changwon National University, Changwon, Republic of Korea
Yogendra Rao Musunuri received the B.Tech. degree in electronics and communication engineering from DPREC, Jawaharlal Nehru Technological University (JNTU), Hyderabad, India, in 2012, and the M.E. degree in image processing and computer vision from Busan University of Foreign Studies (BUFS), Busan, South Korea, in 2017. He is currently pursuing the Ph.D. degree in the control and instrumentation engineering with Changwon National University (CWNU), Changwon, South Korea. His research interests include image processing, computer vision, object detection, super-resolution (SR), advanced driving assistant systems (ADAS), remote sensing, target detection, XAI, edge AI, machine learning, and deep learning.
Yogendra Rao Musunuri received the B.Tech. degree in electronics and communication engineering from DPREC, Jawaharlal Nehru Technological University (JNTU), Hyderabad, India, in 2012, and the M.E. degree in image processing and computer vision from Busan University of Foreign Studies (BUFS), Busan, South Korea, in 2017. He is currently pursuing the Ph.D. degree in the control and instrumentation engineering with Changwon National University (CWNU), Changwon, South Korea. His research interests include image processing, computer vision, object detection, super-resolution (SR), advanced driving assistant systems (ADAS), remote sensing, target detection, XAI, edge AI, machine learning, and deep learning.View more
Author image of Changwon Kim
School of Electrical, Electronics, and Control Engineering, Changwon National University, Changwon, Republic of Korea
Changwon Kim received the B.S., M.S., and Ph.D. degrees in the electrical and electronic engineering from Yonsei University, Republic of Korea, in 2002, 2004, and 2010, respectively. From 2010 to 2015, he was a Senior Researcher with Samsung Electronics, Suwon, South Korea. From 2015 to 2022, he was a Patent Examiner with Korean Intellectual Property Office. Since 2022, he has been with Changwon National University, where he is currently an Assistant Professor. His current research interests include sensor signal processing, image and video processing, and computer vision.
Changwon Kim received the B.S., M.S., and Ph.D. degrees in the electrical and electronic engineering from Yonsei University, Republic of Korea, in 2002, 2004, and 2010, respectively. From 2010 to 2015, he was a Senior Researcher with Samsung Electronics, Suwon, South Korea. From 2015 to 2022, he was a Patent Examiner with Korean Intellectual Property Office. Since 2022, he has been with Changwon National University, where he is currently an Assistant Professor. His current research interests include sensor signal processing, image and video processing, and computer vision.View more
Author image of Oh-Seol Kwon
School of Electrical, Electronics, and Control Engineering, Changwon National University, Changwon, Republic of Korea
Oh-Seol Kwon received the B.S. and M.S. degrees in electrical engineering and computer science and the Ph.D. degree in electronics from Kyungpook National University, Republic of Korea, in 2002, 2004, and 2008, respectively. From 2008 to 2010, he was a Postdoctoral Research Fellow with New York University, New York, NY, USA. From 2010 to 2011, he was a Senior Researcher with the Visual Display Division, Samsung Electronics, Suwon, South Korea. He was a Visiting Professor with Princeton University, Princeton, NJ, USA. He joined Changwon National University, in 2011, and is currently a Professor. His research interests include signal processing, remote sensing, computer vision, deep learning, and human visual systems.
Oh-Seol Kwon received the B.S. and M.S. degrees in electrical engineering and computer science and the Ph.D. degree in electronics from Kyungpook National University, Republic of Korea, in 2002, 2004, and 2008, respectively. From 2008 to 2010, he was a Postdoctoral Research Fellow with New York University, New York, NY, USA. From 2010 to 2011, he was a Senior Researcher with the Visual Display Division, Samsung Electronics, Suwon, South Korea. He was a Visiting Professor with Princeton University, Princeton, NJ, USA. He joined Changwon National University, in 2011, and is currently a Professor. His research interests include signal processing, remote sensing, computer vision, deep learning, and human visual systems.View more
Author image of Sun-Yuan Kung
School of Electrical and Computer Engineering, Princeton University, Princeton, NJ, USA
Sun-Yuan Kung (Life Fellow, IEEE) is currently a Professor with the Department of Electrical Engineering, Princeton University, Princeton, NJ, USA. He has authored or co-authored over 500 technical publications and numerous textbooks, including the VLSI Array Processors (Prentice-Hall, 1988), the Digital Neural Networks (Prentice-Hall, 1993), the Principal Component Neural Networks (Wiley, 1996), the Biometric Authentication: A Machine Learning Approach (Prentice-Hall, 2004), and the Kernel Methods and Machine Learning (Cambridge University Press, 2014). His current research interests include machine learning, data mining and privacy, statistical estimation, system identification, wireless communication, VLSI array processors, signal processing, and multimedia information processing. He was a Founding Member of several technical committees of the IEEE Signal Processing Society. He served as a member for the Board of Governor of the IEEE Signal Processing Society, from 1989 to 1991. He was a recipient of the IEEE Signal Processing Society’s Technical Achievement Award for the contributions on parallel processing and neural network algorithms for signal processing in 1992, the Distinguished Lecturer of the IEEE Signal Processing Society in 1994, the IEEE Signal Processing Society’s Best Paper Award for his publication on principal component neural networks in 1996, and the IEEE Third Millennium Medal in 2000. He was an Associate Editor of VLSI and Neural Networks of IEEE Transactions on Signal Processing in 1984 and 1991, respectively. He was a Distinguished Lecturer of the IEEE Signal Processing Society in 1994. Since 1990, he has been the Editor-in-Chief of the Journal of VLSI Signal Processing Systems.
Sun-Yuan Kung (Life Fellow, IEEE) is currently a Professor with the Department of Electrical Engineering, Princeton University, Princeton, NJ, USA. He has authored or co-authored over 500 technical publications and numerous textbooks, including the VLSI Array Processors (Prentice-Hall, 1988), the Digital Neural Networks (Prentice-Hall, 1993), the Principal Component Neural Networks (Wiley, 1996), the Biometric Authentication: A Machine Learning Approach (Prentice-Hall, 2004), and the Kernel Methods and Machine Learning (Cambridge University Press, 2014). His current research interests include machine learning, data mining and privacy, statistical estimation, system identification, wireless communication, VLSI array processors, signal processing, and multimedia information processing. He was a Founding Member of several technical committees of the IEEE Signal Processing Society. He served as a member for the Board of Governor of the IEEE Signal Processing Society, from 1989 to 1991. He was a recipient of the IEEE Signal Processing Society’s Technical Achievement Award for the contributions on parallel processing and neural network algorithms for signal processing in 1992, the Distinguished Lecturer of the IEEE Signal Processing Society in 1994, the IEEE Signal Processing Society’s Best Paper Award for his publication on principal component neural networks in 1996, and the IEEE Third Millennium Medal in 2000. He was an Associate Editor of VLSI and Neural Networks of IEEE Transactions on Signal Processing in 1984 and 1991, respectively. He was a Distinguished Lecturer of the IEEE Signal Processing Society in 1994. Since 1990, he has been the Editor-in-Chief of the Journal of VLSI Signal Processing Systems.View more

References

References is not available for this document.