Depth Encoding for Neonatal Patient Segmentation

Patient segmentation is an important step in neonatal monitoring for subsequent applications including vital sign monitoring, jaundice detection, or clonic seizure detection. Many studies have faced difficulty in obtaining clear delineation between the patient and the background, instead settling for segmentation of specific body parts, segmentation of visible skin only, or manual selection of regions of interest. The outline of the full body, however, provides a more holistic detection of the entire patient which can be useful for various applications. This study investigates whole-body semantic segmentation of patients under varying levels of coverage (unclothed, clothed, and partially covered with blankets), in complex scenes from the neonatal intensive care unit, from patients placed across all bed types (crib, incubator, and overhead warmer), and in different subject poses (supine, prone, and fetal position). To improve the patient-background delineation, several depth encoding and RGB-D fusion techniques are investigated with a Mask R-CNN model. This study demonstrates how depth information can effectively enhance the RGB image for neonatal patient segmentation, especially when the delineation is unclear due to patient coverage. While RGB images provided suitable predictions for optimal “clear” contours, RGB-D fusion images were required to achieve accurate patient segmentation in challenging “unclear” contours with over 71% IOU and over 82% Dice metrics.