A collaborative distributed multi-agent reinforcement learning technique for dynamic agent shortest path planning via selected sub-goals in complex cluttered environments

Collaborative monitoring of large infrastructures, such as military, transportation and maritime systems are decisive issues in many surveillance, protection, and security applications. In many of these applications, dynamic multi-agent systems using reinforcement learning for agents' autonomous path planning, where agents could be moving randomly to reach their respective goals and avoiding topographical obstacles intelligently, becomes a challenging problem. This is specially so in a dynamic agent environment. In our prior work we presented an intelligent multi-agent hybrid reactive and reinforcement learning technique for collaborative autonomous agent path planning for monitoring Critical Key Infrastructures and Resources (CKIR) in a geographically and a computationally distributed systems. Here agent monitoring of large environments is reduced to monitoring of relatively smaller track-able geographically distributed agent environment regions. In this paper we tackle this problem in the challenging case of complex and cluttered environments, where agents' initial random-walk paths become challenging and relatively nonconverging. Here we propose a multi-agent distributed hybrid reactive re-enforcement learning technique based on selected agent intermediary sub-goals using a learning reward scheme in a distributed-computing memory setting. Various case study scenarios are presented for convergence study to the shortest minimum-amount-of-time exploratory steps for faster and efficient agent learning. In this work the distributed dynamic agent communication is done via a Message Passing Interface (MPI).

I. Introduction

In many surveillance applications, e.g., for the protection of Critical Key Infrastructures and Resources (CKIR), there is a need to monitor larger ground or airborne areas spanning hundreds of thousands of miles. In these applications, where no single system can be sufficient to handle the entire large CKIR systems, there is a need to develop distributed intelligent collaborative geographically and computationally-distributed multi-agent learning system, with a distributed memory setting, where each agent has limited and incomplete knowledge of its environment.