Advancements and Challenges in Video-Based Deception Detection: A Systematic Literature Review of Datasets, Modalities, and Methods

Video-based deception detection has emerged as a promising field that leverages advances in computer vision, machine learning, and multimodal analysis to capture a wealth of nonverbal cues for identifying deceptive behavior. However, the field faces significant challenges related to dataset development, methodological approaches, and ethical considerations. This systematic literature review (SLR) aims to provide a comprehensive analysis of video-based deception detection research, with five distinct contributions: 1) an unprecedented analysis of 21 datasets, revealing critical gaps and opportunities in data resources; 2) a novel evaluation framework for assessing dataset quality and ecological validity; 3) a systematic comparison of multimodal integration approaches, identifying optimal strategies for combining visual, audio, and textual cues; 4) a critical examination of temporal modeling techniques for capturing the dynamic nature of deceptive behavior; and 5) a roadmap for addressing ethical challenges in deployment. Following the PRISMA guidelines, we reviewed studies published between 2019 and 2024 in major databases, including IEEE Xplore, ACM Digital Library, ScienceDirect, and Springer Link. The review process involved a rigorous two-stage screening, which resulted in the inclusion of 42 primary research papers. Our analysis revealed several key findings: 1) only 52.4% of identified datasets are publicly accessible, highlighting a critical gap in research reproducibility; 2) multimodal approaches consistently outperform unimodal methods, with accuracy improvements of 10-15%; 3) deep learning architectures, particularly LSTM variants and attention mechanisms, demonstrate superior performance in capturing temporal aspects of deception; 4) the Real-Life Trial Dataset emerged as the most frequently used dataset (65% of studies), indicating a preference for high-stakes ecologically valid data; and 5) significant ethical challenges remain unaddressed, particularly regarding privacy, bias, and cross-cultural validity. This review makes several novel contributions to advance the field: 1) provides a comprehensive framework for dataset evaluation and development; 2) identifies optimal strategies for multimodal integration and temporal modeling; 3) presents a structured approach to addressing ethical considerations; and 4) offers a detailed roadmap for future research priorities. These contributions will guide researchers in developing more robust, ethical, and generalizable deception detection systems, while addressing critical gaps in current methodologies and datasets.