Unified Test Framework for Voice Enabled Devices

In today's world, testing AI powered voice enabled solutions like Bixby, Siri, Alexa, Google Assistant is highly challenging due to multiple devices (mobile, wearables, TV, AI speaker, refrigerator) enabled voice services, number of natural utterances, different user voice accents are undefined and assuring quality in real user environment (home, office, traffic) and most importantly we need to develop and manage multiple automation tools to solve this problem. In this paper, we proposed a Unified Test Framework (UTF) to handle End to End (E2E) validation of all voice enabled devices in real user environment. Our framework has 4 novel aspects (1) common automation framework for all voice enabled devices (2) framework simulates real user environment to bring high quality (3) framework is platform-independent to support tizen, iOS and android OS platform and (4) framework supports benchmarking test with competitor voice enabled devices, which ensures and improves the product quality in market. The proposed method was applied to Samsung voice enabled devices and played major role in differentiating our quality AI services.