Active Visual Learning for Robots with Dueling Deep Q-Networks and Transformer Encoders

Active vision learning aims to develop intelligent systems capable of actively exploring and understanding their surroundings to optimize detection performance. Although current research has begun to explore how reinforcement learning can drive robots to actively perceive their environment, it often overlooks the critical role of integrating historical contextual information. To address this, we propose an innovative approach to active vision learning, designed to enhance target detection performance in unknown environments. Specifically, this method combines reinforcement learning with deep neural network techniques and cleverly designs a sliding window mechanism to integrate observations over multiple steps into the state input. We employ PointNet++ for feature extraction and utilize a Transformer encoder module to process spatial contextual information, thereby constructing a comprehensive representation of the robot’s environment. Additionally, our carefully designed reward mechanism encourages the robot to prioritize the exploration of diverse object categories. With these reward strategies, we achieve significant improvements in detection accuracy and sampling efficiency. We validated our approach on the real-world 3D dataset R3ED, and the results demonstrate that our method outperforms other baseline methods in terms of performance.