Beat Based Realistic Dance Video Generation using Deep Learning

Deep learning based feature extraction has enabled us to synchronize audio and body movements. It is a promising research field which has great applications in generating sign language, computer animations as well as dance. Previously, computer generated choreography was limited to just stick figure representation. This paper adds image translation technique in dance generation which produces realistic dance moves. With this technique it is possible to produce dance video of a amateur person dancing like a professional. The mapping of stick figure to realistic image is done using Generative Adversarial Network (GAN). We created our own dataset and after adversarial training reconstructed images have SSIM mean 0.864 and LPIPS mean 0.0168. This method produces realistic dance video which is beat based. Body movement speed varies according to the tempo of music which makes it more relevant to real life dance movement.