Hand Gesture Based Region Marking for Tele-Support Using Wearables

Wearable Augmented Reality (AR) devices1 are being explored in many applications for visualizing real-time contextual information. More importantly, these devices can also be used in tele-assistance from remote sites when on-field operators require off-field expert's guidance for trouble-shooting. For an effective communication, touchless hand gestures are the most intuitive to select a Region Of Interest (ROI) like defective parts in a machine, through a wearable. This paper presents a hand gestural interaction method to localize the ROI in First Person View (FPV). The region selected using freehand sketching gestures is highlighted to the remote server setup for expert's advice. Novelty of the proposed method include (a) touchless fingerbased gesture recognition algorithm that runs on smartphones, which can be used with wearable frugal modality like Google Cardboard/Wearality, (b)reducing the network latency and achieving real-time performance by on-board implementation of recognition module. We conducted user studies that suggest the ease and usefulness of the proposed method. Further, we evaluated the the effectiveness of the ROI gesture using the PASCAL Visual Object Classes(VOC) criteria.