ConsistentGS: Modeling Off-Trajectory Driving Scenes With View-Consistent Diffusion Priors

Autonomous driving algorithms rely on high-fidelity closed-loop simulators for safety validation. However, existing scene reconstruction techniques exhibit ghosting artifacts and unseen-view distortion at off-trajectory viewpoints, constraining navigable areas in simulations. To address this limitation, we propose ConsistentGS—a novel driving scene reconstruction framework aimed at expanding drivable regions. Our approach progressively introduces pseudo-views via raycast-based sampling and refines degraded renderings using a diffusion-based restoration network. Directly integrating pseudo-views causes conflicts and haze-like artifacts. We resolve this by introducing a gradient gating module conditioned on multi-view consistency. This mechanism selectively suppresses gradients in certain regions while preserving learning in problematic areas. Building upon OmniRe, ConsistentGS significantly enhances off-trajectory fidelity in driving scenes, enabling closed-loop simulation across expanded navigable regions. Extensive evaluations on the Waymo dataset show that our method outperforms state-of-the-art approaches. Notably, ConsistentGS achieves a 50% reduction in average FID and a 1.2 dB improvement in PSNR over baselines. The code will be open-sourced upon publication.