Voice Command and Hand Gestures for Smart Home

The aim of smart home (SH) applications is to control and monitor home devices in an automated manner. This paper presents a unique approach for SH applications using a multimodal user interface (MUI), which allows users to interact with the system through multiple modes. It is implemented by fusing data from different modalities (text, audio, image, etc.). The fusion of information can occur at different levels, namely the feature, intermediate, and decision levels. This paper introduces an MUI that receives two types of inputs: voice commands and hand gestures. The proposed model relies on a feature-level fusion approach using deep learning. The performance of the proposed model has been evaluated and compared with other works. The results show that the proposed model exceeds other models.