Optimizing Multi-Class Text Classification with Hierarchical Label Filtering and Label Order Analysis

Recent large language models have become popular for achieving state-of-the-art performance in various natural language processing tasks, especially in zero-shot applications where fine-tuning is not required. However, these models underperform in text classification compared to fine-tuned models, due to limitations in reasoning ability and token constraints in in-context learning. Although extensive research explores large language models for text classification, few studies address multi-class classification with these models. This study introduces a new multi-class classification framework using a hierarchical label filtering strategy to manage long prompts in text classification. The influence of label sequence on classification accuracy is further investigated, demonstrating that optimized label arrangements significantly boost performance. Additionally, the study compares the performance of human-defined and model-generated labels in text classification, and analyze performance disparities across models for the same classification task.