Learning Latent Representations for 3D Voxel Grid Generation using Variational Autoencoders

Variational Autoencoders represent a pioneering approach in the field of generative modeling, allowing high-quality image generation and editing. This paper introduces a framework for the generative modeling of 3D Voxel Grids. We utilize a Variational Autoencoder model to generate Voxel Grids by learning well-organized and continuous latent space. One notable advantage of our approach, in comparison to other generative approaches, such as Generative Adversarial Networks (GANs) and Diffusion Models (DMs), is its ability to support a continuous and highly structured latent space, while also requiring less computational resources. We utilize a dataset consisting of 32×32×32 Voxel Grids acquired from 3D meshes via the binvox extension to train our model. As a result, our framework produces reconstructions that are inherently close to optimal. The present study provides quantitative results that demonstrate the training process of the VAE model and the analysis of the latent space. The framework we have developed can serve as an initial reference for generating 3D Voxel Grids and extending traditional VAEs to more advanced techniques, such as Latent Diffusion Models.