Beyond Transformers: fault type detection in maintenance tickets with Kernel Methods, Boost Decision Trees and Neural Networks

The proper handling of customer tickets and main-tenance requests is pivotal for enterprises as it directly impacts customer satisfaction. The ability to rapidly and efficiently react and solve reported issues is in fact a key factor from the customers' perspective, resulting in positive feedback for the company, leading to higher economic and brand-image revenues. The automatic detection of failures from maintenance tickets and support requests can grant faster and more efficient reactions to customers' equipment failures as well as reduced maintenance costs. The analysis of support and maintenance requests is a well-known problem in Natural Language Processing (NLP). State-of-the-art solutions in this field rely on Transformers models, pre-trained on large text corpora, and then fine-tuned on the specific downstream task. However, due to their intrinsic nature, support requests are highly domain-specific and usually similar to short telegraph messages, where the focus is typically encapsulated in short sequences rather than in long dependencies. Hence, ad-hoc methods for pattern recognition might provide comparable performances with respect to Transformers. In this work, two alternative approaches are proposed, based on: Kernel Meth-ods in conjunction with Boost Decision Trees (SpectrumBoost), and Neural Networks for Multiple Representation Learning (DeepMRL). These models have been tested and compared against state-of-the-art models on a real-world set of 131305 maintenance tickets in the Italian language, suggesting that the proposed models outperform Transformers both in the prediction accuracy and in the time and computational resources required for their training.