Perception-based Runtime Monitoring and Verification for Human-Robot Construction Systems

The rising use of robots in construction aims to ease labor-intensive and hazardous tasks. Ensuring safety in human-robot collaboration at construction sites is crucial, necessitating robust safety protocols and smooth interaction. This work aims to develop an open-source framework for monitoring and verifying safety in construction scenarios involving humans and robots. Our proposed framework includes the co-design of two modules: runtime monitoring against Signal Temporal Logic (STL) requirements and real-time reachability analysis using ProbStar. The runtime monitoring module effectively detects, localizes, and predicts human movements within the robot’s operational field. By employing a Kalman filter, we accurately estimate the future paths of workers, which facilitates proactive monitoring of worker safety. This approach enables dynamic adjustments to the robot’s trajectory, guided by quantitatively calculating robustness values of STL specifications in real-time. Our approach leverages real-time data from an RGB-D camera to promptly identify any deviations from expected behavior, further enhancing safety measures. To address uncertainties in localization that make the monitoring results inconclusive for safety judgments, the verification module employs real-time probabilistic reachability analysis to evaluate the likelihood of collisions between robots and obstacles within the robot’s local view. We evaluate the proposed framework across various human-robot interaction scenarios at construction sites.