Post-Lift Analysis of Thermal Imprint for Weight and Effort Detection

Warehousing and manufacturing environments are seeing a proliferation of various body-mounted and environment biosensing mechanisms to assess elements of worker activity indicative of signs of fatigue. In this work, we provide one of the first investigations of thermal imaging technologies to perform automated detection of object weight and subject perceptions of weight using thermal imprints left behind upon interactions with objects of varying masses. We use data recorded from multi-viewpoint thermal cameras when subjects lift and lower weighted cartons with awareness of the weight, as well as blind, i.e., without prior weight knowledge. We demonstrate statistically significant differences across weight classes in mean thermal intensity of thermal imprints on regions of interest extracted around packages in post-lift thermal images for 24 subjects performing 10 randomized interactions each with 0 lb, 15 lb, 30 lb, and 45 lb weights. Using convolutional neural networks trained to be subject independent, we show average accuracies of 75.83% and 70.25% for weight and subject-reported strenuousness detection for weight-blind lifts, and accuracies of 73.84% and 68.92% for weight-aware lifts. Our work plays an important role in strengthening multi-modal algorithms for monitoring workers for signs of fatigue.