A Vision-Based Approach for Controlling User Interfaces of Mobile Devices

We introduce a novel user interface solution for mobile devices which enables the display to be controlled by the motion of the user’s hand. A feature-based approach is proposed for dominant global motion estimation that exploits gradient measures for both feature selection and motion uncertainty analysis. We also present a voting-based scheme for outlier removal. A Kalman filter is utilized for smoothing motion trajectories. A fixed-point implementation of the method was made on a mobile device platform that sets computational restrictions for the algorithms used. Experiments with synthetic and real image sequences show the effectiveness of the method and demonstrate the practicality of the approach in a smartphone.