A Hierarchical Multi-Task and Multi-Agent Assignment Approach: Learning DQN Strategy From Execution

This article investigates the problem of real-time task assignment with heterogeneous agents while considering resource constraints. A hierarchical reinforcement learning-based(HRL) method to address the problem has been proposed. Unlike most existing studies, the method is to assign the tasks to agents such that the resource consumption is minimized while respecting the resource constraints and realizing task distribution balance among agents. At the high level, the real-time task assignment problem within a heterogeneous multi-agent system is formulated as a sequence optimization model which considers multiple constraints. At the low level, a novel reinforcement learning-based hierarchical framework that decomposes the large-scale task assignment problem into the target assignment layer and resource assignment layer is developed. In the first layer, a task assignment strategy closer to the optimal one in real-time by reducing the dimension of the DQN state-action space and learning from execution is produced. In the second layer, a novel resource assignment method is designed to minimize resource consumption and reserve as many agents as possible to handle new tasks by optimizing the resource distribution among agents. Simulation experiments in multi-UAV collaborative emergency material delivery demonstrate that the proposed method can generate high-quality solutions for various problem scales and greatly improve resource balance and real-time performance. Note to Practitioners—This work is motivated by the task assignment problem involving multiple heterogeneous agents in a highly real-time scenario, while taking into account various practical constraints. Each agent can carry various types and quantities of resources within its capacity, and the objective is to minimize resource consumption and travel distance for agents while reserving as many agents as possible to handle new tasks. The proposed approach can be utilized for securing the transportation of valuable assets, pesticide spraying using uncrewed aerial vehicles, and environmental monitoring. The emergency material delivery is used as an illustrative example to demonstrate how UAVs can ensure the rapid transportation of materials and safeguard the lives of affected individuals. To resolve the computationally challenging problem, this paper proposes a HRL method to efficiently solve the task assignment problem among multiple heterogeneous agents, which effectively enhances computational efficiency and improves task completion by optimizing the distribution of resources among agents.