Target Recognition and 3D Pose Estimation Based on Prior Knowledge and Convolutional Neural Network for Robots

In the competition of RoboMaster, the robot needs to trigger the target, the “Energy Mechanism” which consists of nine different dynamic flame numbers, in a nine square area by shooting projectiles. Therefore, 3D target detection should be implemented including target recognition and 3D pose estimation in real-time. As the targets are dynamic flame numbers and quite small in the whole image, it increases the difficulty to detect. The robot should achieve to shoot the target in multi-angle and multi-scale to adjust the competition. To address these issues, we propose a fast and accurate method to detect all nine numbers and estimate each 3D pose based on prior knowledge and convolutional neural network only by a monocular camera. The geometric constraints around the target are employed as prior knowledge when estimating the target pose. Then, we utilize the relative position information to detect the region of each dynamic number in the image, which is recognized by a convolutional neural network trained by flame numbers. Experiments in the actual environment show that our method can achieve the detection of each dynamic number in real-time and high accuracy. The runtime is 29ms on average (about 11ms in detection and 18ms in recognition) and the recognition accuracy is about 94.69%. And our method wins the first place in the technical challenge of 2018 RoboMaster competition.