A novel multimodal gaze-controlled Hindi virtual keyboard for disabled users

Over the last decade, there has been a gradual increase in the number of people with mobility and speech impairments who require novel communication devices. Most of the recent works focus on the Latin script; there is a lack of appropriate assistive devices for scripts that are specific to a country. In this paper, we propose a novel multimodal Hindi language virtual keyboard based on a menu selection with eight commands providing access to spell and type 63 different Hindi language characters along with other functionalities such as the delete command for corrections. The system has been evaluated with eight able-bodied individuals who performed a specially designed typing task. The spelling task has been achieved in three different modalities using: (i) a mouse, (ii) a portable eye-tracker, and (iii) a portable eye-tracker combined with a soft-switch. The performance has been evaluated over the changes that occur with the use of each modality in terms of typing speed and information transfer rate (ITR) at both the command and letter levels for each participant. The average speed across participants with mouse only, eye-tracker only, and eye-tracker with soft-switch were 17.12 letters/min, 10.62 letters/min, and 13.50 letters/min, respectively. The ITRs at the command and letter levels were about 67.58 bits/minute and 62.67 bits/minute, respectively, with only the eye-tracker option. Based on its robustness, the proposed system has the potential to be used as a means of augmentative communication for patients suffering from mobility and speech impairment, and can contribute to substantial improvement in their quality of life.