Development of the Lifelike Head Unit for a Humanoid Cybernetic Avatar ‘Yui’ and its Operation Interface

In avatar-mediated communication, the face-to-face interlocutor must sense the operator’s presence and emotions via an avatar. Although androids resembling humans have been developed to convey presence through appearance and movement, few studies have prioritized deepening the communication experience for operators and interlocutors using android robots as avatars. To address this gap, we introduced the “cybernetic avatar ‘Yui,’” featuring a human-like head unit with 28 actuation points, capable of expressing gaze, facial emotions, and speech-related mouth movements. Through an eye-tracking unit in a head-mounted display (HMD) and the degrees of freedom in both eyes of Yui, operators can naturally control the avatar’s gaze. Additionally, microphones embedded in Yui’s ears allowed operators to hear surrounding sounds in three dimensions, enabling them to discern the direction of calls based solely on auditory information. The HMD’s face-tracking unit synchronizes the facial movements of the avatar with those of the operator. This immersive interface, coupled with Yui’s human-like appearance, enabled real-time emotional transmission and communication, thus enhancing the sense of presence for both parties. Our experiments demonstrate Yui’s facial expression capabilities and validate the efficacy of the system through teleoperation trials, suggesting potential advancements in avatar technology.