Segmentation of the character string pointed out by a finger tip from a pair of color images

We propose a Visual Pointing Out System (VPOS) using a finger tip to indicate a character string from a pair of color images to a computer or a robot. The image pair consists of an image with character strings and a hand, and an image without a hand. This system includes the segmentation of the string regions, the extraction of the hand region, the specification of the finger tip and the extraction of the string region indicated by the finger tip. The algorithm employs morphology (dilation and erosion), a skin color distribution table and geometrical analysis. We have evaluated the effectiveness of this method by computer experiments. Results show that the algorithm can extract both the image of a finger tip from an extended finger, and the character string region indicated by the finger tip. It will be useful and practical as a man machine interface technique.