Reinforcement Learning Based Cardiac Ultrasound Video Summarization Using Weak Supervision and Proximity Reward

Cardiac ultrasound video analysis is a critical part of the diagnostic process for assessing heart failure risk. However, the manual processing of the videos by clinicians is a timeconsuming task, and automated solutions that process each frame for diagnosis are not efficient. While automatic video summarization can pose a solution for these open problems, the lack of annotated data for summarization remains a challenge. Leveraging the Echonet-Dynamic dataset, which provides heart stroke volume information, we formulate a weakly supervised learning strategy. In this study, we propose a Reinforcement Learning based approach for cardiac ultrasound video summarization that utilizes weak supervision and introduces a novel reward function. This proposed reward is based on the volume information, encouraging the reinforcement learning agent to select not only the key frames but also be rewarded for the frames chosen that are close to these frames. In our experiments, with the new reward mechanism, we observed better results in F-score values. Our approach demonstrates the potential of weakly supervised learning in cardiac ultrasound video summarization, addressing the challenges of limited annotated data and contributing to improved cardiac diagnosis using summarized videos.