Enhancing Human-Robot Collaboration in Virtual Reality: A Task-Driven Communication System Using Alexa Voice Service

This paper expands upon an existing Virtual Reality (VR) application designed for the research of Human-Robot Collaboration (HRC) by incorporating a voice-based natural communication backchannel. The core of this VR application revolves around a scenario wherein two teams, each composed of an industrial robot and a human, collaborate to assemble a bicycle. To facilitate collaboration between the human and the autonomous robot throughout the procedure, we implemented the capability for the robot to maintain a conversational flow with its collaboration partner. Derived from the theoretical foundation on how to emulate natural conversations, we designed a clear and deliberate communication system that fits the industrial context and enables the robot to provide guidance for the procedure and explanation for its actions. We identified that Amazon Alexa Voice Service (AVS) presented the most suitable solution for communication integration due to its stability and versatility, albeit with associated privacy and security concerns. The paper describes the implementation of AVS into the VR application. Complications surfaced during the integration of the recommended HTTP/2 client library from AVS into Unity 3D. Overcoming these challenges ne-cessitated disabling name mangling in the C library and configuring specific settings while linking with the MSVC compiler. Apart from the encountered challenges this work outlines potential solutions for future implementations. Furthermore, this industrial HRC-based VR application equipped with bidirectional communication capa-bilities provides the HRC research community with a valuable tool for empirical research. Apart from utilizing the VR application for collaborative interactions between humans and robots in industrial contexts, the hereby presented work can be used for VR application in general to enable natural communication with NPCs and other entities.