Loading [a11y]/accessibility-menu.js
PTP: Parallelized Tracking and Prediction With Graph Neural Networks and Diversity Sampling | IEEE Journals & Magazine | IEEE Xplore

PTP: Parallelized Tracking and Prediction With Graph Neural Networks and Diversity Sampling


Abstract:

Multi-object tracking (MOT) and trajectory prediction are two critical components in modern 3D perception systems that require accurate modeling of multi-agent interactio...Show More

Abstract:

Multi-object tracking (MOT) and trajectory prediction are two critical components in modern 3D perception systems that require accurate modeling of multi-agent interaction. We hypothesize that it is beneficial to unify both tasks under one framework in order to learn a shared feature representation of agent interaction. Furthermore, instead of performing tracking and prediction sequentially which can propagate errors from tracking to prediction, we propose a parallelized framework to mitigate the issue. Also, our parallel track-forecast framework incorporates two additional novel computational units. First, we use a feature interaction technique by introducing Graph Neural Networks (GNNs) to capture the way in which agents interact with one another. The GNN is able to improve discriminative feature learning for MOT association and provide socially-aware contexts for trajectory prediction. Second, we use a diversity sampling function to improve the quality and diversity of our forecasted trajectories. The learned sampling function is trained to efficiently extract a variety of outcomes from a generative trajectory distribution and helps avoid the problem of generating duplicate trajectory samples. We evaluate on KITTI and nuScenes datasets showing that our method with socially-aware feature learning and diversity sampling achieves new state-of-the-art performance on 3D MOT and trajectory prediction. Project website is: http://www.xinshuoweng.com/projects/PTP.
Published in: IEEE Robotics and Automation Letters ( Volume: 6, Issue: 3, July 2021)
Page(s): 4640 - 4647
Date of Publication: 26 March 2021

ISSN Information:

Funding Agency:

No metrics found for this document.

I. Introduction

Tracking and trajectory forecasting are critical components in modern 3D perception systems [1]. Historically, 3D multi-object tracking (MOT) [2]–[4] and trajectory forecasting [5]–[12] have been studied separately. As a result, perception systems often perform 3D MOT and forecasting separately in a cascaded order, where tracking is performed first to obtain trajectories in the past, followed by trajectory forecasting to predict future trajectories. However, this cascaded pipeline with separately trained modules can lead to sub-optimal performance, as information is not shared across two modules during training. Since tracking and forecasting modules are mutually dependent, it would be beneficial to optimize them jointly. For example, a better MOT module can lead to better performance of its downstream forecasting module while a more accurate motion model learned by trajectory forecasting can improve data association in MOT. Our goal is to jointly optimize MOT and forecasting modules and learn a better shared feature representation for both modules.

No metrics found for this document.
Contact IEEE to Subscribe

References

References is not available for this document.