OC-HMAS: Dynamic Self-Organization and Self-Correction in Heterogeneous Multiagent Systems Using Multimodal Large Models

Heterogeneous multiagent systems (HMASs) leverage diverse agent capabilities to address complex tasks in dynamic environments, yet traditional approaches face limitations in autonomy and generalization when adapting to evolving scenarios. To overcome these challenges, we propose OC-HMAS, an IoT-integrated framework that synergizes self-organization and self-correction through multimodal perception. The system processes RGB images, LiDAR point clouds, and instance segmentation maps for real-time environmental awareness, while vision-language models and large language models (LLMs) jointly enable context-aware task decomposition, role allocation, and adaptive planning. Integrated path optimization and obstacle avoidance mechanisms further ensure operational safety and scalability across logistics, inspection, and search-and-rescue operations. Experimental validation demonstrates the framework’s superiority over SMRC-LLM, with 5.15% higher success rates and 14.2% faster task completion in logistics, alongside 4.69% accuracy gains and 12.9% time reduction in inspection scenarios. These results validate its enhanced adaptability in IoT-augmented environments, establishing a new benchmark for autonomous HMAS deployment.