Self-Attention Over Tree for Relation Extraction With Data-Efficiency and Computational Efficiency

Dependency trees parsed from natural language sentences have been proven to be beneficial for the relation extraction task by deep neural networks. However, effectively and efficiently utilizing the structural information of dependency trees remains a challenging research problem for neural networks. Existing methods either struggle to facilitate interaction between nodes at different levels that are distant in the dependency tree or face limitations due to computational inefficiency. In this article, we propose a novel model, named Self-Attention over Tree for Relation Extraction (SATRE), designed for tree-structured data and adept at fully utilizing the structural information implied in the subtrees of dependency trees. SATRE enables interactions among nodes in each subtree, even when they are on widely separated layers. As a result, nodes are learned across multiple subtrees, making SATRE data-efficient; that is, SATRE remains efficient even when data is scarce. Furthermore, SATRE is implemented using a parallel mechanism. Specifically, SATRE parallelly computes node representations of a tree in two levels: across all subtrees of the tree and across all nodes within a subtree. Empirically, SATRE consistently outperforms the compared methods on two real-world benchmarks: the TACRED and SemEval-2010 Task 8 datasets, which shows the effectiveness of SATRE. Meanwhile, extensive experiments indicate SATRE's data-efficiency in utilizing the training data and its computational efficiency in running time.