Usability of Voice-based Intelligent Personal Assistants

Recently the emergence of speech-based Intelligent Personal Assistants (sIPA) has opened a new modality of user interaction with machines. However, till date research in human computer interaction (HCI) has largely focused on graphical user interfaces (GUIs), and not fully facilitate the design of the sIPA's. Therefore, usability of sIPA's is an important area that current research should focus on keeping in mind their commercialization aspect. System usability scale (SUS) has been used by HCI researchers for a long time for measuring the usability aspect and the end-user experiences (UX) in a GUI environment. However, keeping in mind the fundamental differences between a speech-based and a GUI environment, the applicability and performance of SUS in this new context is unknown. In this study a subjective experiment is performed with 61 subjects across some popular commercially available smart-speakers (sIPA's) to investigate into their current state of art usability aspect along with checking if SUS is still a suitable usability evaluation scale in this new scenario. Results suggest that SUS might not be the best scale to use while evaluating the usability of these highly interactive speech-based systems and future research must focus on developing new scales for the current scenario.