Optimizing Heterogeneous Platform Allocation Using Reinforcement Learning

In this paper, we consider the problem of distinguishing a target within an environment using multiple mobile heterogeneous sensing platforms. When used efficiently, the diversity of sensing platforms offers gains over a set of platforms equipped with identical sensors. To optimize over the placement of the platforms, one needs a mechanism for combining the distributed multi-domain sensor data as well as a control to translate that data into coordinated platform movement. Here, we address the latter. By assuming complete sharing of information between platforms, we can establish a baseline for platform movement behavior and add noise incrementally to ensure robustness under noisy communication. The heterogeneity of the platforms removes one axis of symmetry from the problem of mapping platforms over an environment. This means that it is not sufficient to select a set of positions and send platforms based on proximity. We need to select a mapping over all of the platforms so that the platform placed in each location is the one which has the sensor configuration best equipped for information gain there. As each platform accumulates information on the targets within its field of view, we use a modified version of the Upper Confidence Bound algorithm to determine the value of placing each platform in that sector. We also use this algorithm to encourage exploration of sectors which have been unobserved for long periods of time. By assuming random uniform target movement, we can efficiently estimate the environment transitions forward in time. This allows us to generate best trajectories for each platform based on expected target behavior and jointly select their movements. We demonstrate that by framing the problem of distinguishing a target as a partially-observable Markov decision process we can allocate platforms in a way that minimizes search time and displays gains over the same scenario with homogeneous sensing platforms.