A Bio-Inspired Hierarchical Understanding Framework for Satellite Videos Leveraging Low-Rank Representation and Large Language Model

After years of research, moving target tracking in satellite videos has attracted wide attention with various algorithms. However, a big gap still exists between current research and practical satellite video applications with great value. The biggest problem may be the mere trajectories from current methods cannot essentially meet the needs for tasks, and much latent information in video has not been fully exploited. Focusing on this dilemma, we propose a bio-inspired hierarchical satellite video understanding framework (SV-Chat) leveraging LRR (Low-Rank Representation) and LLM (Large Language Model). The LRR is first utilized to separate static backgrounds and moving objects for detailed understanding, imitating the visual pathways and attention mechanism in HVS (Human Vision System). Then, the LLM is enhanced by in-context learning with tailored hierarchical understanding paradigms for satellite videos and conducts corresponding analysis. Finally, the spatial and temporal information from both backgrounds and movements are jointly analyzed for the final report with insights. Experiments demonstrate the feasibility and effectiveness of our framework with a high accuracy of 87.36% compared with ground truth.