Incorporation of Contextual Information into BERT for Dialog Act Classification in Japanese

Recently developed Bidirectional Encoder Representations from Transformers (BERT) outperforms the state-of-the-art in many natural language processing tasks in English. Although contextual information is known to be useful for dialog act classification, fine-tuning BERT with contextual information has not been investigated, especially in head final languages such as Japanese. This paper investigates whether BERT with contextual information performs well on dialog act classification in Japanese open-domain conversation. In our proposed model, not only the utterance itself but also the information about previous utterances and turn-taking are taken into account. Results of experiments on a Japanese dialog corpus showed that the incorporation of the contextual information improved the F1-score by 6.7 points.