Video-based AI Decision Support System for Lifting Risk Assessment

Physical injuries induced by lifting are commonly reported in the workplace. Early risk detection is essential for reducing lifting injuries but requires trained observers to perform assessments manually. Machine learning and computer vision techniques have been proposed to aid ergonomists in lifting risk assessments. However, these methods may not bring the practitioners into the decision-making process and frequently not interpretable to practitioners. We conducted a user study with a proposed risk assessment system that consists of a prediction module, explanation module, and prototype user interface. The prediction module consists of a logistics regression model capable of distinguishing the injury risk levels induced by different levels of force exertion in common lifting tasks. The logistics regression model makes predictions based on explainable body motion, posture, and facial features extracted through computer vision techniques. The explanation module makes up of explainable AI techniques. Specifically, a surrogate model provides local explanations for presenting how the system makes each prediction to users. The prototype interface presents the system’s predictions and explanations. A usability study shows that the proposed system increases crowd-workers’ and domain scholars’ performance in assessing workers’ injury risks in lifting. Furthermore, the usability study also shows that the proposed system increases their confidence in the assessment tasks when the system’s evaluations agree with their subjective evaluations.