System-in-the-Loop Test System With Mixed-Reality for Autonomous Ground Vehicle (AGV) and Military Applications

In this paper, we propose a novel System-In-the-Loop (SysIL) test system based on mixed reality, capable of generating various test cases in outdoor environments, such as rough terrain, for Autonomous Ground Vehicle (AGV) and military applications. SysIL refers to the integration of real systems and simulation environments, enabling more accurate and dynamic testing of system performance across various proving grounds. Implementing a mixed reality test system for military applications presents specific challenges due to the diverse nature of missions and the complexity of unpaved road environments. To address these challenges and develop a SysIL test system for military applications, we propose a novel real-time Light Detection and Ranging (LiDAR) augmentation algorithm and a pose estimation improvement process within a mixed reality framework. The LiDAR augmentation includes real-time occlusion and intensity augmentation algorithms, while the pose estimation improvement process incorporates a real-time algorithm utilizing an Extended Kalman Filter (EKF) with localization sensors (Inertial Measurement Unit (IMU) and odometry) and a 2.5D elevation map. The proposed architecture is designed for an autonomous platform that responds to virtual objects and obstacles generated by the Sensor Signal Generator with Mixed Reality (SSGMR). The SSGMR is positioned between the perception sensors and the controller (either autonomous or manual) of an AGV system, to generate mixed signals for LiDAR point cloud data and camera image data, which are representative sensor data in autonomous applications. The mixed-reality performance of the proposed system is validated through path planning and obstacle avoidance methods within the Robot Operating System (ROS). Finally, several further works are suggested to enhance the performance of the proposed system.