Differentiable Collaborative Patches for Neural Scene Representations

Implicit Neural Representations have been demonstrated to effectively reconstruct various types of data. These networks rely on local structures, such as uniformly distributed grids or quad-trees, to achieve high-quality results. However, due to the nature of these structures, they lack the necessary degree of freedom to achieve optimal layouts. To address this issue, we present a new layout solution for adaptive patches. By allowing local patches to move freely and scale within the scene, we design a differentiable framework that iteratively computes the neural representation and updates the layout of the local patches. Taking into account the coverage, overlap, and balance of the workload, the patches work collaboratively to cover the scene. Our method also supports overlapping patches, which alleviates discontinuous artifacts in the reconstructed image across patch boundaries. Experimental results show that our approach produces high-quality reconstructed images using fewer local patches compared to existing works, demonstrating its effectiveness and efficiency.