IoT-based Leftover Food Segmentation System for Healthcare

Due to declining birthrate and aging population in our society, the number of people in need of care is increasing. Recording the amount of leftover food by care recipients is a critical task; However, due to the shortage of caregivers, automation through IoT-based system is required. This paper proposes a user-friendly leftover detection system using Android device to reducing caregiver burden and improving the accuracy of leftover measurements for monitoring health. This system includes functionalities for easy annotation, training, and inference execution on Android. Additionally, the system is designed to allow the selection of continuous learning as a training method, enabling the model to be trained with newly added data in a short amount of time. The AI model used in this system not only infers leftover areas from input images but also implements algorithms for detecting anomalous inputs. In the experiments, the our system’s inference performance for leftover rice and soup achieved a mIoU of 0.876. And the implementation of continuous learning significantly reduced training time while maintaining model performance, enhancing system efficiency and user-friendliness. In anomaly detection, the system was able to effectively exclude data unrelated to the meal setup. Future work includes incorporating advanced AI models, such as SAM, to assist the annotation process, further alleviating user burden.