Interactively instructing a guide robot through a network

In this paper, we propose a remote-interactive mode for a museum guide robot. In this mode, the remote operator can interact with the robot by using voice and gestures through a network. The operator can instruct the robot what to do using nonverbal behaviors such as touching an object on the display screen while saying an instruction. For example, the operator can ask the robot, “Bring this brochure to him,” while first touching the brochure and then the person on the display. The brochure is detected and tracked using the SIFT feature matching between video camera images. After the robot takes the brochure, the robot detects and tracks the person, and then hands it to him to the person.