ST-KAN: Document Classification Using Sentence Transformer and the Kolmogorov-Arnold Network

The study introduces a novel approach to scientific document classification by integrating Sentence Transformer (ST) embeddings with the Kolmogorov-Arnold Network (KAN). Unlike Multi-layer Perceptron (MLP) with fixed activation functions, KAN utilizes learnable activation functions on edges. In this study, the arXiv Categories dataset is used for evaluation. The dataset contains titles, abstracts of documents, and category labels. The proposed method first merges the title and abstract into a single text representation. This representation is then converted into embeddings using the all-MiniLM-L6-v2 ST. Finally, KAN is applied as the classification model to learn from these embedding representations. Experiments are conducted on the main categories and subcategories to compare performance of ST-KAN against FastText-KAN, FastText-MLP, and ST-MLP. The results show that ST-KAN achieves better performance.