Automatic user state recognition for hand gesture based low-cost television control system

Hand gesture based television (TV) control is attracting more and more researchers. Most of existing works focus on the hand gesture recognition algorithm and the corresponding user interface, while the power consumption or computational cost is not considered carefully. In practice, keeping the camera device and the gesture recognition module running all the time often costs much energy. Till now, few methods have been reported to solve this problem. This paper proposes an automatic user state recognition scheme to recognize the TV user's state and activate the camera-based gesture recognition module only when the user is trying to control the TV. Specifically, the user's behavior active or not is detected by lowcost sensors, the user's gaze watching TV or not is tracked by the face-based view detection, and the user's state is then decided according to a finite-state machine composed of four states: Absent, OtherAction, Controlling, and Watching. The prototypes based on an ultrasonic distance sensor array, a red-green-blue (RGB) camera, and a depth camera are implemented and tested. The results show that the proposed scheme can effectively reduce the power consumption or computational cost of the original hand gesture based control schemes.