Robot-to-Camera Calibration: A Generic Approach Using 6D Detections

For vision controlled manipulation with a robotic arm, knowledge of the camera to end effector transformation is important. This is typically done by classical hand-eye calibration, where a transformation is estimated by observing a known calibration object under different robot poses. The same holds for mobile bases as well, where the mounted cameras' poses with respect to the robot need to be known. However, there are cases where this robot/hand-eye calibration is not possible, for example because of limited movability or reduced precision of the robot. We want to present a new calibration method, which allows for a precise calibration also in these situations. This is achieved by rotating the robot around one axis and sampling a high number of visual features, in our case AprilTag markers. By fitting a Bingham distribution to the sampled markers around several configurations of the robot and determining the rotation center and axis, a mathematical model connecting the kinematics of the robot to the camera frame of reference can be found. We show that this method can be applied to cases where a classical hand-eye calibration fails, e.g. a robot base with limited movability. Our method allows an estimation of the camera to robot transformation even under those challenging conditions, e.g. where no part of the robot is visible from the camera.