Human Computer Interaction Advancement by Usage of Smart Phones for Motion Tracking and Remote Operation

With the advent of faster home computers, growth and development of smart phones combined with advancements in computer vision, recent years have seen various research, homebrew and commercial projects aimed at enhancing human computer interaction by novel means. These methods involve both live video processing and transport of user operation to devices external to the computer. Various methodologies of object recognition, tracking and processing have been used in wide practical applications like traffic analysis, pedestrian tracking, gesture processing and mouse emulation. This paper presents a comprehensive and integrated method to emulate movement and standard features of a computer mouse in various scenarios of different lighting conditions and means of operation using OpenCV, a computer vision library, cvBlob, a blob analysis library, and B4A, a BASIC compiler and IDE for Android. The paper also compares operating success efficiencies in different operating conditions. A standard computer webcam is used to observe and track the user's smartphone screen for movement and preprogrammed events signaling clicking, dragging or other operations. Unlike contemporary human computer interaction systems, the key advantage of this system is the non-requirement of any kind of network that connects the handheld device to the computer to achieve remote operation.