Integrative Human-Computer Interaction System Using Eyes and Facial Movement

This study investigates the possible improvement of communication systems for individuals with motor impairments, such as ALS, by the incorporation of real-time facial movement detection and eye tracking. We propose a system that combines webcam-based facial movement identification with an eye-tracking virtual keyboard, enabling users to customize input methods according to their physical ability. Seven healthy participants evaluated the system while seated in a single position. We evaluated essential criteria like input velocity, accuracy, and user comfort, and integrated a feature to gauge pain severity through facial expressions to enhance emergency situation prediction. The research demonstrate that participants exhibited varied preferences for facial characteristics, with improved performance noted when employing their preferred landmarks rather than non-preferred ones. This study implies that, when implemented for patients in an incomplete locked-in state, the system may increase communication and provide a customized solution aligned with each individual's preferences. The findings demonstrate that the use of real-time facial movement detection alongside eye-tracking enhances communication for individuals with severe motor impairments, providing a more efficient, customized, and practical alternative to traditional approaches.