Hand and Object Segmentation from Depth Image using Fully Convolutional Network

Semantic segmentation is an important step for hand and object tracking as subsequent tracking algorithms depend heavily on the accuracy of the segmented hand and object. However, current methods for hand and object segmentation are limited in the number of semantic labels, and lack of a large scale annotated dataset to train an end-to-end deep neural network for semantic segmentation. Thus, in this work, we present a framework for generating a publicly available synthetic dataset, that is targeted for upper limb rehabilitation involving hand-object interaction and uses it to train our proposed deep neural network. Experimental results show that even though the network is trained on synthetic depth images, it is able to achieve a mean intersection over union (mIoU) of 70.4% when tested on real depth images. Furthermore, the inference time of the proposed network takes around 6 ms on a GPU, thus making it suitable for real-time applications.