A Review of Video Generation Approaches

Generating videos from some initial frames is an appealing field of research in deep learning. There exists an ever expanding foray of approaches to generate long-range and realistic video frame series. Generating videos can help predict trajectories and even model object movements, to enhance autonomous robots. However, there are only a few comprehensive studies that review various approaches on the basis of their relative advantages, disadvantages, and evolution. Hence, this paper presents a detailed overview of Deep Learning based approaches employed to tackle the complex problem of video generation. The approaches involve Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs) and even the Transformer model. Finally, the performance of all the approaches are examined and compared on the BAIR Robot Pushing dataset.

I. Introduction

Video generation has gained substantial traction as a research topic. Researchers all over the world are working on many different methods to generate long range, high resolution, indistinguishably natural videos. The popularity is due to many critical advantages attached to it. Prominent fields that have great relevance to video generation are reinforcement learning [1] and motion planning of autonomous systems [2]. Although it is an interesting task, it is a very difficult problem to model, owing to its multi-modal nature and the exponentially growing tree of possibilities after the passage of initial frames. Video generation has numerous real-world applications which include interpolation of full sized videos from cropped videos and scene generation in video games. Further, it can also be utilized for generating videos which can serve as datasets for machine learning models. In robotics, video generation can play a vital role in handling occlusion, predicting object movement [3] and trajectory optimization.