Multimodal user interaction framework for e-commerce

E-commerce has grown to be a major user of e-services and online purchases through e-commerce are largely preferred over the traditional brick and mortar purchasing. Yet it is challenging for the consumers to fully experience the products or services with limited senses, lack of tangibility and sense of presence. Therefore, the objective of this study is to assess how multimodal interactions can be used in e-commerce, to improve the consumer experience. This research describes the design and implementation of multimodal interactions enabled prototype framework for e-commerce developed to address that issue. The framework supports three interaction modes: speech inputs, gesture inputs and interactive 3D product presentations. A case study and a usability study conducted on this framework showed satisfactory results and also revealed some interesting social and cultural barriers to multimodal interactions that involve language and psychological factors.