A Real-Time Chinese Food Auto Billing System Based on Instance Segmentation

Recently, food segmentation has been a hot topic in both academia and industry. Solutions for western food segmentation have been proposed and the performance is promising which meets the requirements in scenarios like diet managements and calorie estimation. Inspired by these achievements, we decide to design a Chinese food Price automatic billing system based on instance segmentation methods. However, how to segment Chinese food remains a challenge due to the wide variety of ingredients and cook styles. It's impossible to collect sufficient images to train a segmentation model to detect all kinds of potential Chinese food. To overcome these issues, rather than detecting each singular dish, we reformulate the task by segmenting a set of selected plates with Chinese food, we propose a FoodSyn module, which synthesis images by cropping food part in UECFoodPIX and pasting them on plates images. The generated images are then fed into encoder-decoder network to segment instances for training. Extensive experiments show that our proposed approach performs well in practical scenarios with mIoU over 950/0. The fps is over 20 when deployed on OnePlus 9 Pro. Codes will be released once the paper is accepted.