Post-Trained Language Model Adaptive to Extractive Summarization of Long Spoken Documents

The General Meeting Understanding and Generation Challenge challenge track 2 in ICASSP 2023 Signal Processing Grand Challenge aims at extractive summarization of meeting transcripts. The main characteristic of this challenge is abridged as long spoken documents that are extremely difficult to manipulate with pre-trained language models. In this paper, we propose a post-trained DeBERTA which does not only adapt to spoken language but also manages long documents. According to the experimental results, the proposed method shows a 6% higher test score than the challenge baseline model.