Video Understanding based Multiple Entity Collaborative Scheduling for Social Governance

In the scenarios of social governance, it often requires real-time handling of various unexpected events, which usually involves collaboration among multiple entities. The keys to this task are real-time action and the coordinated scheduling of diverse public departments. To address this issue, we propose a video-based collaborative scheduling solution. By leveraging real-time video streams captured by on-site cameras, we can instantly access live footage. Utilizing the powerful video understanding capabilities of large multimodal models, we are able to effectively interpret on-site event information. Additionally, by considering the functions and locations of various departments along with pre-prepared response plans, we enable collaborative scheduling through Retrieval Augmented Generation (RAG) capabilities. We conduct experiments on a set of real event video data and invite experts to evaluate the generated collaborative scheduling solution. Results demonstrate that the scheduling proposals achieved over 90% satisfaction among experts.