ICanDo: Intellectual Computer AssistaNt for Disabled Operators

The paper describes a prospective multimodal system ICanDo (Intellectual Computer AssistaNt for Disabled Operators) developed in SPIIRAS and intended for assistance to persons without hands or with disabilities of their hands or arms in human-computer interaction. This system combines the modules for automatic speech recognition and head tracking in one multimodal system. The architecture of the system, methods for recognition and tracking, multimodal information fusion and synchronization, experimental conditions and obtained results are described in the paper. The developed system was applied for hands-free work with Graphical User Interface in such tasks as Internet communication and work with documents.