RealityAvatar: Comprehensive Head Avatar Generation with 360° Rendering

We present RealityAvatar, a novel model for 360°-renderable head avatar generation. RealityAvatar supports both 3D Gaussians and textured polygen 3D mesh as representation. A multi-stage framework is proposed, beginning with a pretrained 3D GAN generator that produces multi-view images from random noise, text, or image inputs, which are then used as training data. 3D Gaussians are bound to initialized FLAME faces, with their scales and rotations determined by the corresponding faces. FLAME parameters and vertex positions are optimized for a detailed mesh consistent with the FLAME topology. Then, the mesh and images are refined via a differentiable renderer to produce texture maps. Finally, 3D Gaussians undergo unconstrained optimization for enhanced rendering quality. To the best of our knowledge, RealityAvatar is the first generative model for 360°-renderable and animatable human heads. The experimental results show high rendering quality and robust drivability.