iRay: Mobile AR Using Structure Sensor

Using depth information has become more popular in recent years, as it adds a new dimension to 2D camera views. We have developed a novel mobile application called iRay, which uses depth information to achieve highly accurate markerless registration on a mobile device for medical use. We use a Structure Sensor to capture depth information that is portable to iPad. Its SDK also provides SLAM data to track pose. ICP is applied to achieve highly accurate registration between the 3D surface of a human torso and a pre-scanned torso model. The experiments demonstrate our results under motion blur, partial occlusion, and small movements. This application has the potential to be used for medical education and intervention.