4DRT-SLAM: Robust SLAM in Smoke Environments using 4D Radar and Thermal Camera based on Dense Deep Learnt Features

LiDAR- and Visual- SLAM are prone to fail in smoke and fog environments, because LiDAR and visual camera cannot penetrate smoke and fog. Fortunately, both 4D radar (x, y, z, velocity) and thermal camera can work robustly in such adverse conditions. However, little related work has been performed. In this paper, 4DRT-SLAM is presented, where 4D radar and thermal camera are fused to perform the SLAM in smoke and fog conditions. Since the point cloud of 4D radar is rather sparse and noisy, it is difficult to extract valid geometric features (line and plane). In 4DRT-SLAM, firstly, we perform 2D feature extraction and matching on thermal images. Considering that hand-crafted features do not work well for thermal images, we use deep-learned features (LoFTR). Secondly, the corresponding 3D coordinates of these feature points are obtained by projecting the point cloud of 4D radar onto thermal image, with the intrinsic and extrinsic parameters. Thirdly, Perspective-n-Point (PnP) is performed to calculate the transformation matrix between two consecutive frames. Experiments are performed in real smoke environments. Both quantitative and qualitative analyses demonstrate that 4DRT-SLAM is effective and robust in smoke environments.