An Efficient Multi-Modal Image Registration Method for Intelligent Acupuncture Robot

This invention presents a multimodal image registration method for intelligent acupuncture robots, equipped with imaging devices such as the ultrasound and optical tracker. These multimodal images provide distinct information in various formats, but bring challenges for acupuncturists in image interpretation and processing. To address this, a multimodal image registration method is developed based on acupoint structure and acupuncture planning, aiming to align images of different formats to produce comprehensive images of acupoints’ epidermal surfaces. To enhance the accuracy of needle insertion monitoring and assessment, the proposed method employs rigid multimodal image registration, integrating ultrasound, optical tracking, and depth camera technologies. Given the point sets from ultrasound, optical tracker, and depth camera images, two rigid transformations map the ultrasound and optical tracker point sets onto the depth camera coordinate system. The rigid registration assumes a single transformation matrix for all points in the same image, optimizing alignment with the target point set. Since ultrasound provides 2D cross-sectional images and depth cameras and optical tracker yield 3D data. The method first positions the ultrasound’s 2D cross-section in 3D space using the needle insertion point and robot needle tip coordinates. Feature extraction from ultrasound images is followed by real-time matching and registration. Upon successful registration, the needle tip position is tracked across all modalities.