Effects of variability in synthetic training data on convolutional neural networks for 3D head reconstruction

Convolutional neural networks have recently shown great success in computer vision. They are able to automatically learn complicated mappings, often reaching human or super-human performance. However, a lack of labeled data can preclude the training of such networks. This is the case in the reconstruction of 3-dimensional human heads from 2-dimensional photographs. Approaching the problem backwards, starting from 3-dimensional heads and using photo-realistic rendering, one can create any number of training data to tackle the problem. This way, fine control over the data allows for new insights into how a convolutional neural network interprets data and how variability in the training and test data affect its performance. We perform a systematic analysis in order to determine how the presence of different types of variability in the training data affects the generalization properties of the network for 3-dimensional head reconstruction.