Generating Pandas Code for Big Table Question Answering Using Large Language Models

Currently, large language models are extensively utilized across various domains to address a wide range of challenges. Such models exhibit the capability to process data in diverse formats effectively, including tabular data. This paper describes a method for generating pandas code to answer queries related to big tables. The approach relies solely on specific elements of the table - namely, column names and sample rows - as input data for the model. The performance evaluation employed the DataBench dataset, which provided questions encompassing both numeric and categorical answers. The research contrasted the GigaChat Max and GPT-4o-mini models; GigaChat Max achieved an accuracy of $\mathbf{7 8. 6 \%}$, whereas GPT-4o-mini demonstrated an accuracy of 87%. The GPT-40-mini model attained state-of-the-art performance in numeric answers (88.5%) and categorical answers (86.6%) as per the DataBench benchmark. The experiments indicated that an increase in the number of input rows could adversely impact model performance. The findings of this study may contribute to enhancing the efficiency of analytical systems that incorporate large language models.