Audio-Visual Deepfake Detection System Using Multimodal Deep Learning

With the rise of deepfake videos in today’s digital landscape, there is a pressing need to develop advanced detection and mitigation technologies. Deepfake videos, which use machine learning algorithms to manipulate and alter audiovisual content, can spread false information, create confusion, and even cause harm. To address this issue, our work leverages Deep Learning technology to process audio-visual data in real-time through a web interface, specifically a browser plugin. Our approach uses a multimodal neural network that is fed extracted audio and visual features from a video for deepfake prediction. The model achieves a maximum validation accuracy of 90 percent and is used to build an API that is responsive, and low-latency. Our end-to-end solution is implemented as a Chrome extension using JavaScript that communicates with the API. With this solution, we aim to contribute to the development of advanced deepfake detection and mitigation technologies that can help prevent the spread of false information and its potential consequences.