Reacting Like Humans: Incorporating Intrinsic Human Behaviors Into NAO Through Sound-Based Reactions to Fearful and Shocking Events for Enhanced Sociability

As robots increasingly enter human-centered environments, the ability to exhibit natural, emotionally resonant behaviors becomes crucial for effective interaction. In this work, we present a multi-modal system designed to enable the NAO robot to sense its environment, react naturally to sudden loud sounds with human-like fear responses, and locate the source of the sound. The system comprises a motion generator, a sound classifier, and an object detector. For motion generation, we developed a deep network-based model to synthesize diverse movements. Sound detection leveraged a transfer learning model utilizing spectrograms of sound signals, while object detection employed a YOLO model trained specifically for this task. These individual components were integrated into a comprehensive “fear” module, which was implemented on the NAO robot. The module’s performance was evaluated through practical testing, during which both experts and non-experts in robotics completed a questionnaire on the robot’s performance. Results demonstrated that the proposed system effectively convinced participants that the NAO robot could perceive and respond to its environment with human-like reasoning and behavior. Additionally, findings revealed that non-experts held higher expectations regarding the performance of social robots. Given our promising results, this preliminary exploratory research provides a fresh perspective on social robotics and could be a starting point for modeling immediate, emotion-driven human behaviors in robots.