Hierarchical Attention Model for Acquiring Relationships Among Sentences

In this paper, we propose a hierarchical attention model for summarization. Normally, sentences have relations among another sentence and it is important to consider these relations in summarizing. Our proposed model can make each sentence vectors from document composed of multi sentences and get relations among sentences from these vectors by the incorporated operation. As an operation of taking relations, we use self-attention and gated convolutional neural network. It has been reported that these operations can get dependencies among words, and self-attention is particularly powerful. Therefore we adopted these operations expecting the same work in sentences. We conducted an experiment of title generation by using Japanese news articles. We evaluated the performance of our proposed model by Rouge and visualized the relations among sentences.