Interpretable Video based Stress Detection with Self-Refine Chain Reasoning

Stress detection is critical for mental and physical well-being, yet traditional methods such as self-reports and physiological sensors face limitations in efficiency and scalability. Video-based stress detection, leveraging visual cues learned from an annotated video database, offers a non-invasive, cost-effective alternative. However, most models function as black boxes, lacking transparency in their decision-making process, which hinders their trustworthiness. To address this, we propose an interpretable video-based stress detection model that incorporates Chain-of-Thought (CoT) reasoning of large foundation models. Our model follows a structured reasoning chain “Describes Assess-e-Highlight”, mimicking the decision process of psychology experts. To further enhance model reliability, we integrate a self-refinement mechanism that allows the model to reflect on and improve its predictions using Direct Preference Optimization (DPO) to ensure accuracy and faithfulness. Experimental results on two video-based stress detection datasets demonstrate that our approach outperforms state-of-the-art models in both accuracy and interpretability. We release our code at https://github.com/debby1103/stressdetection.git.