Detecting Anomalous Robot Motion in Collaborative Robotic Manufacturing Systems

Anomalous robot motions caused by cyber attacks and inherent defects can lead to task failures as well as harmful accidents in collaborative human–robot workplaces. External Internet of Things (IoT) sensors are essential for reliably monitoring robot tool paths and detecting deviations as early as possible, especially in anomalous situations where the robot system and its internal sensors may not function as expected. However, affordable external IoT sensors may suffer from poor-quality measurements, necessitating data-driven algorithmic enhancements. In addition to external sensors providing independent monitoring, to effectively detect trajectory deviations, we need anomaly detection methods that can capture complex dynamic patterns in the nonlinear trajectory time-series data. In this work, we propose a framework to accurately estimate robot trajectories during normal robot operations as well as to detect various types of anomalous trajectory changes using an external camera. The proposed framework efficiently incorporates marker-based pose estimation, long short-term memory (LSTM), and residual control charts. The framework was evaluated in a shared human–robot assembly task. The results show that we can accurately estimate robot trajectories by enhancing camera-based measurements. Moreover, it effectively detects anomalous trajectory changes in their early stages. The motion deviation upon detection assists in determining a safe working distance. The framework is also generalizable to previously unseen trajectory deviations and allows the use of a variety of IoT sensors.