Efficient Integration of Neural Representations for Dynamic Humans

While numerous studies have explored NeRF-based novel view synthesis for dynamic humans, they often require training that exceeds several hours, limiting their practicality. Efforts to improve training efficiency have also encountered challenges because it is hard to optimize non-rigid transformations, thus leading to coarse renderings. In this work, we introduce an innovative approach for efficiently learning and integrating neural human representations. To achieve this, we propose a comprehensive utilization of the features stored in both canonical and observational spaces, facilitated through a collaborative refinement process that integrates canonical representations with observational details. Specifically, we initially propose decomposing high-dimensional multi-space feature volume into several feature planes, subsequently utilizing matrix multiplication to explicitly establish the correlations between different planes. This enables the simultaneous optimization of their counterparts across all dimensions by optimizing interpolated features, efficiently integrating associated details, and accelerating the rate of convergence. Additionally, we use the proposed collaborative refinement process to iteratively enhance the canonical representation. By integrating multi-space representations, we further facilitate the co-optimization of multiple frames’ time-dependent observations. Experiments demonstrate that our method can achieve high-quality free-viewpoint renderings within nearly 5 minutes of optimization. Compared to state-of-the-art approaches, our results show more realistic rendering details, marking a significant advancement in both performance and efficiency.