Frequently Asked Question Retrieval Problem: A Text Classification Approach

Since even Large Language Models, which are the cutting edge of technology today, require knowledge grounding systems, business-specific Frequently Asked Questions (FAQ) recognition systems are becoming increasingly important. In real-life scenarios, FAQ sets are meticulously hand-crafted by experts and cover a wide range of topics; hence their effective use produces high value. In this study, despite its origins in recommendation systems, this FAQ recognition problem was approached as a few-shot text classification problem. K-Nearest Neighbor Classifier (KNN), Multi-Layer Perceptron (MLP), which are basic text classification methods with very similar equivalents in recommendation systems, and SetFit, which is an advanced method of contrastive learning, were compared on two of the best-known FAQ datasets, FAQIR and StackFAQ. In the case of 5 samples per class; it was seen that the most successful method was MLP, with accuracy scores of 97% and 80% in the FAQIR set, and 99% and 95% in the StackFAQ set; in English and Turkish respectively. Both Turkish and English syllogisms are presented using Query-Question (q-Q), query-Answer (q-A) and query-Question+Answer (q-QA) similarities.