Online Reinforcement Learning for Autonomous Sensor Control

For a heterogeneous sensor grid to achieve situational awareness over an area of interest and successfully detect, track, classify, and identify (DTCI) objects of interest, individual sensor platforms must be able to autonomously adapt to changing environments by modifying their controllable parameters in real-time. To this end, we propose a novel methodology for active sensor management by means of online reinforcement learning that simultaneously learns an optimal policy and discretizes the state-action space of interest through dictionary learning - learning of a finite set of representative state-action space. This method features minimal a priori assumptions on the state-action space, allowing for gradual adaptation of the policy to occur as a sensor observes new information. The major innovation is the use of dictionary learning to gradually determine the individual basis functions whose linear combination approximates the Q-function. We show in several physics-based photorealistic simulations, based on a real-world test and experimentation location represented in the Gazebo framework, that our methodology is capable of learning near-optimal policies while taking significantly less training time than is traditionally found in other reinforcement learning methods. These simulation results capture performance of several independent sensors monitoring the same area of interest with small amounts of field of view overlap. They set the foundation for future work where sensors interact to coordinate their adaptation to further enhance the resultant situational awareness.