Automatic Database Alignment Method to Improve Failure Data Quality

Life-cycle data analysis mainly relies on coherent failure data. Generally, in the industry, failure data are recorded in various Enterprise Resource Planning (ERP) databases and can lead to inconsistent failure data analysis. The aim of this paper is to propose a new method to automatically merge existing failure datasets into one main database to reduce inconsistency and allow valuable analysis and stochastic/predictive models. The methodology is as follows. First, the data is cleaned using rules from domain knowledge. Second, similar features between the two databases were evaluated by applying correlation technique. Finally, we developed a rule-based engine to merge existing failure datasets into one main database. Our rule-based engine was calibrated according to the distribution of the observed distance between similar features. At first, we were able to match the failure data by 65%, then we enhanced our algorithm using Natural Language Processing (NLP) technique and we were able to reach an outstanding score of 85%. Traditionally this data alignment is done yearly by human experts. This is an extremely time-consuming task, and our automatic matching algorithm reduces workload by 70% and biases in life-cycle data analysis. With a confidence level our algorithm interactively suggests matches according to their distances.