EnableTech: A Multimodal Gesture and Voice-Controlled Home Automation System for Enhanced Accessibility

Providing access in smart homes is important to people with disabilities. This article introduces EnableTech, a home automation system utilizing multimodal interface voice and gesture recognition for error-free, hand-free control over home appliances. The system is based on VC-02 AI Thinker to process voice commands in real-time and an ESP32 microcontroller to perform the operations effectively. For gesture recognition, a CNN model is trained on the Sign Language MNIST dataset using OpenCV and the MediaPipe Hand Detector to precisely recognize hand gestures through a laptop camera. Robust error handling, adaptive command mapping, and real-time execution with an average response time of 0.5 seconds are the notable innovations. Experimental results show 94% accuracy in gesture recognition and 98% reliability in command execution, providing a seamless user experience. The system improves significantly mobility-impaired people's independence and accessibility while providing a price-effective and scalable smart home solution. Future development will focus on enhancing multi-gesture recognition, environment adaptability, and IoT integration for potential applications.