Automatic Procedure Following Evaluation Using Petri Net-Based Workflows

An operating procedure (OP), also known as checklist or action plan, is a list of actions or criteria arranged in a systematic way, commonly used in areas such as aviation or healthcare to ensure the success of critical tasks and to help decrease human errors. In these areas, operators are hardly trained to follow the OP carefully, but the evaluation of how they are following, it is usually performed manually by an expert instructor. Automating this evaluation process would lead to an objective and scalable analysis of the operator performance, which is extremely important in areas where the number of operators to evaluate is high. This problem, which can be referred as automatic procedure following evaluation, needs of new techniques and formalizations due to current conformance checking methods do not fit well with some aspects of a OP. In this paper, OPs are modeled as Petri Net-based workflows, and interact with the data log of the system to allow an automatic evaluation of the progress and time spent following the OP. In order to illustrate the contributions of this paper, a case study is carried out designing and modeling an emergency OP for an unmanned aircraft system, and evaluating the proposed approach with a battery of tests.