Automated Descriptive Audio Generation for Visually Impaired Individuals: A Framework Integrating Object Detection, Emotion Analysis, and Speech Synthesis

This study introduces an automated framework aimed at enhancing video accessibility for visually impaired individuals by generating descriptive audio. Utilizing advanced tools such as PySceneDetect for extracting keyframes, YOLOv8 for detecting and classifying objects, and DeepFace for analyzing emotional expressions, the system identifies key visual elements and emotional contexts within videos. Detailed captions are created using the BLIP model and are subsequently transformed into natural and customizable audio using ParlerTTS. The resulting descriptive audio is synchronized with the original video through tools like MoviePy and PyDub. This integrated solution provides an efficient, scalable, and automated method to make video content more accessible, enabling visually impaired users to better interpret and enjoy visual media.