The Use of Domain Knowledge Models for Effective Data Mining of Unstructured Customer Service Data in Engineering Applications

Despite the fact that enterprises are routinely collecting massive amounts of data from customers, only a relatively small body of knowledge engineering (KE) work has addressed methods and application of KE to the design, development, and maintenance of engineering systems and products. A major challenge when applying KE to such applications is that the data is often unstructured and in the form of text exchanges between the customer and the enterprise. While the importance of modelling domain knowledge in order to produce meaningful results from mining unstructured data has been recognized, most approaches are based primarily on the linguistic structure of the text and keyword taxonomies. These approaches share the common issue that the knowledge extraction results are often not properly structured for solving the engineering problem of interest and, therefore, require manual post-processing before they can be applied. Our hypothesis is that the a priori modelling of the engineering problem of interest is crucial for both (1) efficient (rapid) collection, representation, and structuring of domain knowledge, and (2) the proper integration of domain knowledge with analytical KE methods in order facilitate the extraction of useful knowledge. In order to validate our hypothesis, we apply this approach to the important real-world engineering problem of monitoring the occurrence of product failure modes, and thereby product quality, using customer support cases. In order to translate the free-form text provided by the customer into engineering failure modes we use two methods from engineering design, the Function Analysis System Technique (FAST) and Failure Modes and Effects Analysis (FMEA), to provide the necessary domain knowledge model. This model then drives the collection, representation, and structuring of the failure modes for the product of interest. These failure modes are used as the class labels when applying data mining classification techniques (e.g., Support Vector Machine) to the support case data. The labelled support case data then can be aggregated by failure mode in order to compute a number of failure mode metrics that can be used to monitor product quality. We have demonstrated our approach to monitor the quality of a network security product at a large computer networking company using a data set of 100,000 customer support cases.