TabNet for high-dimensional tabular data: advancing interpretability and performance with feature fusion

TabNet is a novel deep learning architecture proposed to overcome the limitations of traditional deep learning models in progress with tabular data, including diverse features, sparse datasets, and interpretability. With the capabilities of sequential attention mechanisms and selective feature sparsity, an optimal balance between accuracy and clarity can be achieved, enabling feature transformation that outperforms any model like XGBoost in revealing complex feature interactions. It has been proved flexible in many various fields such as finance, healthcare, energy prediction, environmental monitoring, etc. However, the TabNet exhibits several weaknesses, including its computational intensity, sensitivity to noise, and its limited compatibility with heterogeneous/multi-modal systems. The paper presents a Feature Fusion methodology that embeds TabNet's intermediate attention outputs with the initial attribute set in order to create a more complete and robust representation of the data. The fusion boosts TabNet's ability to catch and handle anomalies and outliers, especially in the financial forecasting scenarios where substantial market shocks and extreme events cause significant changes in predictions. By employing a real-world dataset about the stock market as a benchmark, our experiments demonstrate that TabNet + Feature Fusion significantly enhances performance metrics mainly in the context of Root Mean Squared Percentage Error (RMSPE) which is highly sensitive to outliers. The findings confirm that this actual approach not only addresses the limitations of traditional TabNet but also provides a powerful tool for tackling complex, high-dimensional, and noisy tabular data in various critical domains as well as finance.