Tracking and detection of pointing gesture in 3D space

In this paper, we discuss pointing gesture estimation in the context of Human Robot Interaction (HRI). Pointing gesture provides and utilizes information in Kukanchi (Interactive Human Space Design and Intelligence) more natural than other conservative method like keyboard or mouse. Therefore, we design a user interface adopting pointing gesture to manipulate objects and robot naturally in 3D space. Initially, human posture is tracked. Later, the information is used to detect the dynamic pointing gesture using our proposed method which is Hidden Markov Model (HMM). Preliminary experiment shows the average angular error of pointing direction is less than seven degrees.

1. Introduction

To realize a living environment where human interact with robot in their daily life, we need to provide the information from user to the system and vice versa. Kukanchi is one approach where the living space is designed with many sensors and computers so that the environment is ubiquitous (Fig. 1). In this living space, the necessary software and hardware are connected by wireless and/or wired network that no one will notice their presence thus providing robotics service to human. Hence, it is important to have a user interface to input information from user to system and output the feedback from system to user.