Modular Gaussian Splatting: Instance Decomposable Learning and Adaptive Rendering of 3D Scenes via Mixture of Experts

This paper introduces Modular Gaussian Splatting (Modular-GS), a novel method that leverages 3D Gaussian Splatting and Mixture of Experts (MoEs) for decomposing and representing 3D scenes as a combination of Instance Gaussians. Modular-GS achieves scene decomposition by inputting multi-view data and automatically generated masks, facilitating fine-grained modeling of individual objects. Our approach enables controllable editing and dynamic rendering of the scene by selectively combining different experts, supporting instance insertion across datasets. Experimental results demonstrate that Modular-GS improves scene modeling quality and efficiency, offering new possibilities for Radiance Field Rendering. This work extends the boundaries of 3D scene representation and editing, advancing techniques for semantic understanding and real-time rendering. Our code and models will be at https://modular-gs.github.io/.