Developing a Cooking Robot System for Raw Food Processing Based on Instance Segmentation

This study presents an autonomous cooking robot system developed to improve culinary tasks through the classification and individual grasping of primary food materials. Our focus is on the recognition and manipulation of fried chicken parts and raw shrimp, which are essential in various culinary preparations, particularly frying. To distinguish and segment a specific target from a mix of similar objects, we utilize a Mask Region-based Convolutional Neural Network (Mask R-CNN) algorithm. Moreover, our robotic system incorporates a pose estimation technique to handle food materials of varying shapes. This system addresses the use of a direction vector transformed to determine 3D poses in real world, enabling a two-finger cooking robot to accurately grasp soft food materials. We have performed real robot experiments to demonstrate the system’s ability to handle both fried chicken pieces and raw shrimp, verifying that our proposed method is effective. Additionally, we have confirmed the accuracy of our image segmentation approach.