TPU Based Deep Learning Image Enhancement for Real-Time Point-of-Care Ultrasound

Handheld ultrasound devices are becoming more prevalent in point-of-care ultrasound workflows. However, these devices are computationally constrained which challenges the advances in deep learning methodology for real-time use on mobile Point-of-care ultrasound (POCUS) devices. In this work, we explore the feasibility of running MimickNet, a deep learning clinical post-processing model, on Tensor Processing Units (TPUs), hardware designed for deep learning operations capable of running on only 2 watts of power at 1.8 V with a form factor of 10 mm x 15 mm. We show that real-time deep learning based post-processing is feasible at 20 - 120 FPS for 1472x160 to 224x224 axial sample x B-mode scan line configurations. We refer to the TPU based model as MimickNet Mobile. MimickNet Mobile achieves outputs nearly identical to the original MimickNet with a structural similarity index measurement (SSIM) of 0.98
$\pm$
0.001 and a mean squared error (MSE) of 0.0001
$\pm$
0.0 over our test set of 588 frames consisting of 168 phantom frames and 420 prospectively acquired human liver frames. We investigate the latency of other common mobile architectures such as separable convolution. Finally, we investigate the distribution of model parameter error when quantizing MimickNet
$\mathtt{float32}$
weights to MimickNet Mobile
$\mathtt{int8}$
weights. This work demonstrates that real-time POCUS deep learning image enhancement is feasible using TPUs. Future ultrasound device manufacturers can consider incorporating a TPU for the added flexibility of supporting several deep learning architectures without compromising on power management and form factor.