Enhancing Ultrasound Training with AI-Driven Real-Time Feedback

Ultrasound is central to cardiology for its safety and accessibility, but image acquisition remains operator-dependent and limited by workforce shortages. Simulation training can help, though most systems rely only on visual cues without explanatory feedback. We introduce a simulation framework that combines vision-language and reinforcement learning models to support training in transesophageal echocardiography (TEE). The system uses keyframes as navigation waypoints and provides real-time multimodal guidance through text and visuals.In a user study with 15 participants of varying ultrasound familiarity, all successfully acquired standard two- and four-chamber views, with 100% success rates. Participants identified language instructions and keyframes as the most useful guidance tools. These findings demonstrate the potential of AI-driven multimodal feedback to enhance simulation-based echocardiography training.