Agreement Study Using Gesture Description Analysis

Choosing adequate gestures for touchless interfaces is a challenging task that has a direct impact on human-computer interaction. Such gestures are commonly determined by the designer, ad-hoc, rule-based, or agreement-based methods. Previous approaches to assess agreement grouped the gestures into equivalence classes and ignored the integral properties that are shared between them. In this article, we propose a generalized framework that inherently incorporates the gesture descriptors into the agreement analysis. In contrast to previous approaches, we represent gestures using binary description vectors and allow them to be partially similar. In this context, we introduce a new metric referred to as soft agreement rate (SAR) to measure the level of agreement and provide a mathematical justification for this metric. Furthermore, we perform computational experiments to study the behavior of SAR and demonstrate that existing agreement metrics are a special case of our approach. Our method is evaluated and tested through a guessability study conducted with a group of neurosurgeons. Nevertheless, our formulation can be applied to any other user-elicitation study. Results show that the level of agreement obtained by SAR is 2.64 times higher than the previous metrics. Finally, we show that our approach complements the existing agreement techniques by generating an artificial lexicon based on the most agreed properties.

I. Introduction

Gestural modalities offer an intuitive and natural mode of interaction with machines that resembles human-to-human communication. Nowadays, gesture-based interfaces are part of handheld devices, smart TVs, autonomous vehicles, and gaming consoles [1], [2]. Furthermore, several orthodox disciplines such as surgery have benefited from such interfaces, since it allows surgeons to control medical systems in a touchless manner while maintaining total asepsis [3]–[5].