Evaluating Feasibility of Image-Based Cognitive APIs for Home Context Sensing

Cognitive API is API of emerging AI-based cloud services, which extracts various contextual information from non-numerical multimedia data including image and audio. Our interest is to apply image-based cognitive APIs to implement smart and affordable context sensing services in a smart home. However, since the existing APIs are trained for general-purpose image recognition, they may not be of practical use in specific configuration of smart homes. In this paper, we therefore propose a method that evaluates the feasibility of cognitive APIs for the home context sensing. In the proposed method, we exploit document similarity measures to see how well tags extracted from given images characterize the original contexts. Using the proposed method, we evaluate practical APIs of Microsoft Azure, IBM Watson, and Google Cloud for recognizing 11 different contexts in our smart home.