DeepFake Detection by Prediction of Mismatch Between Audio and Video Lip Movement

Deepfakes are synthetic falsified media that are altered to create convincing imitations of a person that are not authentic. The primary technique used to create deepfakes, uses training generative neural networks such as autoencoders and Generative Adversarial Networks(GAN's). This form of realistic imitation and recreation of the audio or video of a person poses a security and privacy risk. Most current existing research does not assess the audio and video of a Deepfake simultaneously, rather they consider them independently. This research aims to bridge the gap presented and suggest a detection method that uses both audio and video of the deepfake to detect it. The proposed approach attempts to discern situations where the visible motions of a person's lip are not in accordance with the audio that is being played, which could mean that the content has been altered or manipulated. Further, this paper suggests that with greater fine tuning, the exact time lag between audio and lip movement could be estimated.