A Method of Relation Extraction: Integrating Graph Convolutional Networks, Relative Entity Position Attention and Back-Multi-Head-Attention Mechanism

Relation extraction is the task of identifying entity pairs and relationship between entity pairs. Moreover, it is a key method for knowledge discovery. It has been proven effective to run the graph convolutional networks on the syntactic dependency tree to extract the syntactic features of sentences. The entities in the sentence and position of entities have a great influence on the results of relation extraction. Based on graph convolutional networks, we propose a relation extraction model that combines relative entity position attention and Back-Multi-Head-Attention mechanism. On the one hand, we design a network structure based on the relative entity position attention mechanism, so that the model can focus on the position of the entity pair in the sentence. On the other hand, we propose the Back-Multi-Head-Attention mechanism, which enables the model to maintain the memory of the original features while extracting features. It prevents the degradation of the network. We integrate the outputs of the relative entity attention mechanism network and the outputs of GCNs, then process the feature by Back-Multi-Head-Attention mechanism and classifier to extract the relation. The experiments are conducted on the sentence-level relation extraction. Experiments show that our approach achieves competitive F1 value compared with previous dependency-based models.