Using Artificial Intelligence in Mining Excavators: Automating Routine Operational Decisions

Worldwide mining operations, which account for nearly 11% of global power consumption, are 28% less productive today than a decade ago. Meanwhile, mobile equipment was present in nearly 40% of mining fatalities and more than 30% of injuries in 2017. Building intelligence into existing mining excavators improves the safety, productivity, and energy efficiency of mining. This can provide perception, monitoring, and control capabilities that produce accurate, actionable data for mines. The intelligent excavator has an in-cab monitor that provides real-time status updates and guidance to operators as well as a remote monitoring portal. Multiple sensors, including a rugged camera that overlooks the excavator bucket, high-resolution surveillance cameras, radar, arm geometry, hydraulic pressure monitoring, and electric motor power measurement, sensors are integrated. Additionally, a set of human labeled video frames is used as training inputs to train an artificial neural network (NN) to perform multiple object localization via an optimization process, which (combined with other sensory data) is used to monitor the wear and breakage of sacrificial ground engaging tools (GETs), detect foreign objects, analyze the size distribution of the material inside the bucket, measure the bucket payload, and augment the operator's skill and experience. This information is vital to mining operations aiming to optimize dig, load, and dump cycles for energy consumption, downtime, and operator efficiency. Aside from improving operational efficiency, intelligent excavator solutions enable us to develop highly perceptive shovels with decision-making modules that pave the way for fully autonomous excavator operation.