Joint lesion detection and classification of breast ultrasound video via a clinical knowledge-aware framework

Ultrasound is an important routine screening modality for breast cancer. Breast ultrasound screening is a dynamic process, and clinical practice involves radiologists recording representative frames during dynamic breast scanning for subsequent diagnosis. However, existing computer-assisted diagnosis methods often concentrate on dull diagnostic results by analyzing these representative frames and ignore the valuable information in the dynamic examination process that facilitates diagnosis. Moreover, breast lesions could exhibit various characteristic differences during scanning, and effective learning of lesion representations is challenging and may affect the clinical interpretability of the methods. To this end, we draw insights from the behavior of radiologists during the dynamic breast examination and leverage the knowledge of breast anatomy to propose a clinical knowledge-aware framework for lesion detection and classification of breast lesions in ultrasound videos. It is equipped with global-local attentive aggregation and a dynamic allocation mechanism that simulates the behavior of radiologists searching for diagnostic clues, thus integrating local localization and global semantic information from the video into the feature representation of the lesion. An anatomically-aware transformer is also designed to refine the lesion feature representation using spatial relationships within and across different anatomical layers of the breast anatomy. Extensive experiments show that the proposed framework can achieve competitive performance in both lesion detection and video classification tasks while exhibiting good clinical availability and interpretability, with an average precision of 40.80% and an AUC of 85.86% on our constructed breast video dataset and an average precision of 39.79% and an AUC of 87.04% on a publicly available dataset.