Modeling Physical Activity in Children by Combining Raw Hip-Worn Accelerometry, 2D Pose Estimation, and Direct Observation

This work presents an inductive search of physical activity intensity features for developing computer vision data sets used in automatic physical activity observation systems. An online survey was conducted to calibrate the ground truth definitions in a data set of video synchronized with accelerometers. Experts from the research field were asked to classify 24 short video samples of children’s physical activity relative to three metabolic equivalence of task units – the presumed threshold of moderate to vigorous physical activity. Leave-one-out disagreement analysis was applied until moderate agreement was achieved (12 respondents remain, Light’s
$\kappa $
= 0.62). The predictive power of features from hip-worn ActiGraph wGT3X-BT 30Hz raw accelerations and several freely available 2D pose estimation models are explored by cut-point analyses and logistic regression while using several approaches to account for uncertainty. Features from the acceleration- and video pose estimation domains are combined in correlation analyses in a twelve-hour data set representing relatively unstructured behavior from 24 children ages 8–13 filmed in four different locations. Results indicate that changes in the hip angles of pose-estimated kinematic skeletons across 10 fps video frames can supplement or possibly substitute accelerometers for estimating physical activity intensity in uncrowded indoor scenes. When taking such an approach to labeling physical activity recognition data sets, good joint tracking capacity of the pose estimation method should increase the robustness of the hip angle features.