A Two-Stage System for Spoken Language Understanding

In this paper, we propose a Two-Stage system for SLU, which consists of Automatic Speech Recognition (ASR) tasks and Natural Language Understanding (NLU) tasks. Our work is done in the context of an ICASSP 2023 Spoken Langauge Understanding Grand Challenge. In the first stage, we use a model based on the encoder-decoder structure to recognize the speech utterance into text. In the second stage, we combine Bidirectional Encoder Representations for Transformers (BERT) and Conditional Random Field (CRF) to train intent determination and slot filling jointly. In addition, we compress the data using Byte Pair Encoding (BPE) and construct the BERT word list by pre-training to save a large number of parameters, thus extending the model depth and obtaining an Exact Match boost. The final experiments demonstrate the effectiveness of our proposed model.