Generation of Indian sign language by sentence processing and generative adversarial networks

This paper proposes generating Indian Sign Language from sentences using sentence preprocessing and Generative Adversarial Networks (GANs). The proposed method focuses on converting sentences to brief notations in the form of glosses and generating synthetic video frames for each gloss using a Generative Adversarial Network (GAN) architecture, thus resulting in a video representation for the input sentence. The glosses are mapped to a corresponding minimal skeletal representation, which is used as an input and a base image for the GAN model. The GAN outputs the corresponding video frames, which are further processed to produce a video. The dataset consists of video representations of words present in the Indian Sign Language. The proposed paper measures the performance of the method by using the SSIM index and a side-by-side comparison of real and synthetically generated video frames.