Auxiliary Label Classification Based Multi-Label Limb Movement Recognition of Preterm Infant

Limb movement recognition of preterm infants (PI-LMR) in neonatal intensive care units (NICUs) is important for infant health monitoring. However, little attention has been paid to intelligent PI-LMR. Due to the weak correlation among limb movements of preterm infants, the various limb movement combinations and imbalanced data distributions are the main challenges of PI-LMR. To address these issues, a novel multi-label limb movement recognition (MLLMR) algorithm with a dual-branch structure and multi-label fusion loss is proposed. The various movement combinations can be decomposed into limbs thanks to multi-label learning. Particularly, the multi-label fusion loss consisting of the binary cross entropy (BCE) and the pairwise ranking loss (PRL) is proposed to optimize the probabilities to the ground truth labels and the ranking between positive and negative labels, simultaneously. The weighted fusion loss is further developed to address the imbalanced label distributions. Subsequently, an auxiliary task for the classification of zero-, single- and multi-limb movements is constructed to constrain the feature space of primary task for better multi-label learning. Experiments on real clinical preterm infants video dataset from Jiaxing Maternity and Child Health Care Hospital are conducted and the results demonstrate the effectiveness of the proposed algorithm.