Augmentation of a Virtual Reality Environment Using Generative Adversarial Networks

Building a dataset to train machine and deep learning models has become a challenging task ever since more complex architectures and deeper neural networks have begun to be utilized with higher frequency, especially across computer vision applications in various fields. This hardship is prompted by the lack of data readiness and the difficulty associated with gathering enough of it, in order to avoid unbalanced datasets and overfitting during training. Occasionally, simulation environments are employed as a result of the unavailability of time and resources to collect real-world data. We present an application of Generative Adversarial Network (GAN) data augmentation in a vision-based planetary rover localisation application. Due to the absence of extra-terrestrial data, the system was trained on several artificially-built lunar terrains inside a Virtual Reality (VR) simulation environment. In order to further enhance our dataset, a GAN was utilized to augment the data retrieved from the simulation, while simultaneously averting a reduction in image quality, which is common with other forms of data augmentation. Despite the fact that GAN was trained with only a small number of images, it was able to recreate areas of the VR environment with high precision. Additionally, training the system with the GAN enhanced data improved the output compared to employing only basic data augmentation techniques.