Deep Learning Video Classification of Lung Ultrasound Features Associated with Pneumonia

Ultrasound (US) imaging holds promise as a low-cost versatile, non-invasive point-of-care diagnostic modality in low-and middle-income countries (LMICs). Still, lung US can be challenging to interpret because air bronchograms are anechoic and the US images mostly contain artifacts rather than lung anatomy. To help overcome these barriers, advances in computer vision and machine learning (ML) provide tools to automatically recognize abnormal US lung features, offering valuable information to healthcare workers for point-of-care diagnosis. This paper describes deep learning algorithms that target three key US features associated with lung pathology: pleural effusion, lung consolidation, and B-lines. The algorithms were developed and validated using a large and varied dataset of 22,400 US lung scans (videos) from 762 patients of all ages (newborn to adult) in Nigeria and China. The architectures include effective methods for leveraging frame-level and video-level annotations, are light enough to deploy on mobile or embedded devices and have high accuracy (e.g., AUCs ≈ 0.9). Coupled with portable US devices, we demonstrate that they can provide expert-level clinical assistance for diagnosis of pneumonia, which is the leading cause of both childhood mortality and adult hospitalization in LMICs. We also discuss some of the challenges associated with determining ground truth for pneumonia, which impact the question of how to leverage ML models for lung US to support clinical diagnosis of pneumonia.