Interactive Visual Procedure using an extended FMEA and Mixed-Reality

In this paper, an application is created for the HoloLens, a headgear that enables users to experience mixed reality. The application proves that it is plausible to create an interactive visual guide in the form of a Unity application, in order to support operators with the fault detection assessment of a machine. This application utilizes WorldAnchors and a Vuforia engine to fix the holograms spatially. It shows the steps to solve detected faults through holograms and CAD models. The troubleshooting steps are modelled using an extended Failure Mode and Effects Analysis (FMEA) as a knowledge extraction method. This method extracts expert knowledge from the operators in a systematic way, as well as it maps the faults with the current feed of the machine data. Users can interact with buttons through the guide on the HoloLens until the fault is solved. The fault detection system is located in a software framework, which collects data from the machine using the OPC-UA communication protocol. This system is proven to be useful, as well as it has a low latency of 51 ms.