Measuring Motion of Deformed Surfaces for Soft-bodied Robots/Animals with Multi-colored Markers

This paper presents a 3D multi-colored direction-detection marker and the shape estimation algorithm designed to sense the motion of complex deformed surfaces for a soft-bodied robot. The proposed approach utilizes the area ratio of the four colors captured by a single camera to detect the marker's direction. By attaching many of these markers to the surface of a soft robot or an animal, we measure the complex deformation and motion of a soft robot/animal with the proposed algorithm using the image processing library, OpenCv. The results of three experimental use cases demonstrate that the measurement system successfully and easily captured the motions containing multi-modal deformations of a soft-bodied caterpillar-like robot, a live fish, and two fingers of a gloved hand. The proposed method may be utilized to design sensor-rich feedback controllers.