In-Context Vision-Pattern-Language Model for Enhancing Vessel Activity Explanation

Illegal vessel activities pose significant threats to marine resources and ecosystems worldwide, necessitating effective monitoring and detection methods. Current vessel monitoring systems struggle to accurately interpret vessel behaviors at a detailed level due to limitations in utilizing multi-modal data and regulatory frameworks. To overcome these challenges, we propose a new Vision-Pattern-Language (VPL) model designed to enhance the explanation and detection of illegal, unreported, and unregulated (IUU) vessel activities. To handle AIS-off in boundary waters, proposed model integrates satellite imagery with Automatic Identification System (AIS) trajectory data using a probabilistic fusion approach based on Maximum A Posteriori (MAP) estimation with Monte Carlo dropout. Additionally, the VPL model employs a CLIP-based zero-shot classifier to accurately identify vessel behaviors. To support law enforcement, the VPL model with in-context learning also generates faithful and contextually reasonable explanations grounded in the fused data and a legal-text database. Extensive experimental evaluations on the AIS and satellite imagery dataset demonstrate that the VPL model significantly improves trajectory prediction accuracy and classification performance than baselines. Moreover, VPL attains higher faithfulness and reasoning scores compared to Llama-3.3, highlighting its potential for robust and reliable maritime surveillance and contributing meaningfully to the detection and regulation of IUU vessel activities.