Interactive Machine Learning on Edge Devices With User-in-the-Loop Sample Recommendation

Interactive machine learning (IML) aims to make machine learning an easy-to-use tool for novice users to solve personalized tasks. However, despite the recent popularity of edge AI, research into interactive machine learning on edge devices has not been conducted actively. Existing IML designs cannot be directly applied to small edge devices due to interface and computational resource limitations. In this paper, we propose a method for efficient model personalization on a small interactive object recognition camera device by combining sample recommendations with an IML workflow. The proposed method recommends training data candidates from unlabeled samples in addition to the usual annotation operations. Our method interactively trains a noise filter to handle a noisy sample pool obtained while using the device. The user can indicate whether the recommended sample corresponds to 1) the recommended class; 2) other classes; or 3) noise unrelated to the recognition task by providing ternary feedback. Our system is designed to gradually update both the target classifier and the noise filtering recommendation modules on the basis of feedback. We show that our feedback design achieves more efficient model training while improving system usability through a systematic evaluation and user study using a prototype device.