Imperio: Cyber Foraging Framework for On-Device Inference based Mobile Applications

Machine learning has become almost commonplace in almost all industries and every aspect of human life. Beyond that on-device AI has become a watershed in machine learning allowing devices to run machine learning models on the device. On-device AI guarantees low latency, high reliability, and more security when compared with cloud based approaches. Even through flagship devices are released every year, majority of the people lean towards mid-range devices due to budget constraints. So, in the market the majority of the people are using older mid-range devices. This is where cyber foraging becomes useful which allows such devices to offload heavy computational tasks to powerful devices in the vicinity. This paper presents Imperio, a cyber foraging framework for on-device inference based Android mobile applications. It allows on-device inference based mobile applications to offload machine learning tasks to resource rich devices. Further, it has a decision making engine which decides when to offload. Results show that there is a significant reduction in task execution time. Imperio is a novel approach to on-device inference based Android mobile applications which use cyber foraging to reduce latency.