Task–Driven Image–to–Image Translation for Automotive Applications

Image-to-image translation attempts at mapping an image from a given domain into a different domain. In this paper we introduce a novel image-to-image translation paradigm, where the translation itself is performed by a system of generative adversarial networks (GANs), guided by a set of pretrained networks, called task networks. The underlying hypothesis is that these task networks can guide the translation process of the GAN generator to focus on those aspects of the image which matter the most for the tasks in question. Moreover, our work is motivated by a real-life situation, created by the need to adapt existing, duly trained networks to work on images acquired with a different sensor than the one used to acquire the images used to train the networks. In order to optimize the image domain translation from the new sensor to the old one, we use the output loss of the trained neural networks to guide the training process of the image-to-image translation engine, thus forcing it to create images which improve the performances of the trained, guiding networks. Experimental results presented therein confirm the usefulness of our approach for an automotive application within the context of driver monitoring systems. More specifically we show that guiding the learning of the image-to-image translation module with the loss of a fixed neural network trained to estimate the driver eyelid opening, improves the performances of the latter when being fed with images output by the translation module.