Domain Translation via Latent Space Mapping

In this paper, we study the problem of multi-domain translation: given an element (a) of domain A, we wish to generate a corresponding element (b) in another domain B, and vice versa. Acquiring supervision in multiple domains can be a tedious task, also we propose to learn this translation from one domain to another when supervision is available as a pair (a, b), and leverage possible unpaired data when only (a) or only (b) is available. We introduce a new unified framework called Latent Space Mapping (LSM), which exploits the manifold assumption to learn a latent space from each domain. Unlike existing approaches, we propose to further regularize each latent space using available domains by learning each dependency between pairs of domains. We evaluate our approach on three tasks performing i) a synthetic dataset with image translation, ii) a real-world task of semantic segmentation for medical images, and iii) a real-world task of facial landmark detection.

I. Introduction

In many machine learning tasks, different modalities can be modeled as different domains, and different data can be modeled as different views of the same reality. For example, in the context of an autonomous vehicle, the RGB camera, the depth map, and the segmentation map can be considered as three views of the same reality. Often one domain can be considered as an input domain, e.g. a CT (computed tomography) scan of a patient, and other domains as an output domain, e.g. the organ segmentation of the scan. Here we do not consider an input nor an output domain, but we would like to learn to translate from any domain to another one. This definition masks the classical definition of an input and output domain, since any domain can be a possible input or output. While the rising amount of available data has brought great results in domain translation tasks in a fully supervised fashion, in some fields the quantity of available supervision is limited and hard to obtain, as in the medical field where a domain expert is required to create a hand-made segmentation [1], [2], and this low supervision setting often reduces the performance of classical deep learning models.