Anomaly Detection and Interpretation from Tabular Data Using Transformer Architecture

Anomaly detection in tabular data are often needed in most domains, including finance, medicine, and engineering. However, traditional methods typically require intensive processing and often lack interpretability, which limits their practicality. As the complexity and volume of tabular data increase, there is a need for advanced techniques that can handle diverse data types while offering interpretable results. We introduce a novel approach for anomaly detection of tabular data and its interpretation. The anomaly detector employs a transformer model with a custom embedding layer tailored for tabular data. While the model’s attention weights do not directly explain its decision-making process, they offer useful insights when interpreted thoughtfully. Our anomaly interpreter uses these attention weights to identify irregularities by comparing patterns in anomalous data against those in normal data. When the model flags a row as anomalous, the interpreter analyzes its columns and assesses their relationships using a reference association matrix created from the normal dataset. Any deviations from expected associations are flagged as potential rule violations, highlighting unusual relationships between columns. We evaluate our approach against baseline anomaly detection techniques, including Multi-Layer Perceptrons (MLPs), Long Short-Term Memory (LSTM) networks, One-Class Support Vector Machines (OC-SVM), and Deep Support Vector Data Description (Deep-SVDD). Our experiment uses labeled datasets from the Outlier Detection DataSets (ODDS), and KDD datasets, assessing performance using standard metrics alongside a novel mutation analysis technique. Results indicate that our approach achieves accuracy comparable to or surpassing existing methods while also offering interpretable explanations for detected anomalies.