Investigating Pretrained Language Models and Learning Methods for Vietnamese Text-to-SQL

Text-to-SQL is one of the most challenging tasks in Natural language processing (NLP), which automatically constructs an executable Structured query language (SQL) query from a given database schema and natural utterances. Current datasets like Spider bring more challenges for the NLP community by considering the domain shift among databases, especially in low-resource languages such as Vietnamese. One of the main current approaches is finetuning pretrained language models (PLMs), which leverages the language knowledge encoded in PLMs to adapt to a new specific task. In this paper, we investigate both the effects of different PLMs, including monolingual and multilingual PLMs, and the impacts of learning methods on the Vietnamese Text-to-SQL task. We also proposed a simple but effective learning approach to address the domain shift problem in the Vietnamese Spider dataset. The experimental results show that our method gives competitive results compared to other strong baselines.