On Coupling Classification and Super-Resolution in Remote Urban Sensing: An Integrated Deep Learning Approach

Motivated by the state-of-the-art optical sensing and image processing technologies, remote urban sensing (RUS) has emerged as a powerful sensing paradigm to capture abundant visual information about the urban environment for intelligent city monitoring, planning, and management. In this article, we focus on a classification and super-resolution coupling (CSC) problem in RUS applications, where the goal is to explore the interdependence between two critical tasks (i.e.,
classification
and
super-resolution
) to concurrently boost the performance of both the tasks. Two fundamental challenges exist in solving our problem: 1) it is challenging to obtain accurate classification results and generate high-quality reconstructed images without knowing either of them
a priori
and 2) the noise embedded in the image data could be amplified infinitely by the complex interdependence and coupling between the two tasks. To address these challenges, we develop
SCLearn
, a novel deep convolutional neural network architecture, to couple the classification task with the super-resolution task in an integrated learning framework to
concurrently
boost the performance of both the tasks. The evaluation results on a real-world RUS application over two different cities in Europe (Barcelona and Berlin) show that SCLearn consistently outperforms the state-of-the-art baselines by simultaneously achieving better land usage classification accuracy and higher reconstructed image quality under various application scenarios.