GT&I GAN: A Generative Adversarial Network for Data Augmentation in Regression and Segmentation Tasks

For data augmentation (DA), Generative Adversarial Networks (GANs) are typically integrated with CNNs or MLPs to generate samples in classification and segmentation tasks. For classification, categorical ground truth is leveraged in conditional GANs to generate samples for each class. For regression, data generation becomes complex as the aim now is to generate both the samples (images) and their continuous ground truth vectors. GANs for classification can no longer, therefore, be leveraged for DA on regression. To address this issue, we propose GT&I_GAN, a novel GAN-based DA model that generates jointly image samples and their ground truth continuous vectors by learning their conjoint distribution. The main idea behind GT
$\mathbf{\& I-GAN}$
is to add, to the RGB sample image, an additional (fourth) channel associated with the ground vector. GT&I_GAN offers the great advantage of generating conjointly the samples and their ground truths by a single model without needing an additional network. We assess our approach on an image dataset where the ground truth consists of a high dimensional vector of continuous values. The results show that the synthetic data consisting of the image & ground truth vector pairs are realistic and allow improving the CNN regressor performance. Moreover, we show that our GT&I_GAN can be leveraged seamlessly for segmentation tasks by adding, in a similar way, the ground truth segmentation mask as an additional channel to the input RGB image.