Diff-Refiner: Enhancing Multi-Agent Trajectory Prediction with a Plug-and-Play Diffusion Refiner

The inherent stochasticity of the agents' behavior presents a challenge to trajectory prediction models, which are required to generate multiple plausible future trajectories. Recently, diffusion models have been applied to implement multimodal trajectory prediction. Existing approaches typically employ a standard diffusion process, denoising from a sample drawn from a Gaussian distribution. However, we identify that most agents exhibit an obvious movement trend, rendering many initial denoising steps redundant-primarily transitioning from pure noise to an initial coarse trajectory. To conquer this challenge, this paper innovatively proposes a diffusion refiner that can be used along with existing multi-agent trajectory prediction models to improve their performance. Specifically, we first leverage a baseline model for predicting the coarse future trajectory. Then, the diffusion model is applied as a refiner to reduce the prediction error. Moreover, our method is naturally plug-and-play, allowing convenient integration with existing models. To achieve this, we improve the traditional diffusion process to not only converge towards noise but also the coarse predictions from the baseline model. In such a case, standard step-skipping sampling techniques is inapplicable and we further propose an ordinary differential equation (ODE)-based fast sampling method. Extensive experiments with selected baseline models demonstrate the effectiveness of our approach.