Exploring Techniques for Photo-realistic Image Generation from 3D Models - A Deep Learning Approach

The product companies are going towards Digital Transformation for digital customer experience. This is effective for marketing and sales specially during the pandemic. For the product visualization, it is necessary to create multiple 2D views of a product which is a cumbersome and time-consuming process especially when there are copious number of products. Traditional rendering techniques and image augmentation for dataset generation either take quite a lot of time or do not yield photo-realistic images. The recent progress in generative models and 3D deep learning provide a promising avenue in this regard. This paper explores different techniques for rendering photo-realistic 2D images from 3D CAD models. This study compares the performance of three different models pix2pix, Pytorch3D and SynSin based on three different approaches namely image-to-image translation, mesh rendering, and view synthesis respectively to generate photo-realistic images. The models are tested using the dataset provided by Schneider Electric. Results demonstrate that Pytorch3D is the better model to generate photo-realistic images. These approaches can emerge as the initial steps for digital twin technology.