LLM-Guided Multi-Object Tracking: Enhancing Retail Insights with TempRMOT

This paper introduces an innovative system that integrates advanced multi-object tracking with intelligent linguistic query processing to address the challenges of dynamic retail environments. By combining the Temporally enhanced Referring Multi-Object Tracking (TempRMOT) framework with a Large Language Model (LLM), our system provides real-time tracking and practical insights. It seamlessly integrates visual and linguistic data to solve key challenges, such as distinguishing employees from customers, monitoring uniform compliance, and analyzing customer behaviors. Experimental results on custom retail datasets demonstrate significant performance improvements, achieving 81.82% Exact Match and 84.5% semantic similarity in query responses. This integration not only enhances tracking accuracy but also establishes a new paradigm for adaptive retail management, paving the way for more intelligent and context-aware systems.