Efficient and Effective INR: A Dive into Levels-of-Experts (LoE) and Sine Activation

Implicit neural representations have emerged as a transformative approach in machine learning and computer graphics, enabling the generation and manipulation of intricate data domains including images, 3D models, and physical simulations. However, scaling INRs to large datasets or high-resolution signals poses significant challenges due to the exponential growth of network parameters with input space size. In this study, we introduce a method based on Levels-of-Experts (LoE) for implicit neural representations that proves to be more efficient and promising. We have identified that the activation associated with LoE results in sharp discontinuities, which in turn diminish the performance of the implicit neural representations. To tackle this problem, we substituted the Leaky ReLU activation function with the Sine activation function, without the need for any additional initialization schemes. This straightforward yet potent approach surpasses many previous techniques and is capable of representing large datasets or high-resolution signals effectively.