DoreBer: Document-Level Relation Extraction Method Based on BernNet

Document-level relation extraction (RE) task aims to predict predefined relation types of every entity pair in a given document. Compared with the sentence-level counterpart, document-level relation extraction task requires reasoning in a more complex environment, where exist much longer text and much larger amount of entities, making it a more challenging task. However, previous methods suffers from over-smoothing problem when the count of GNN layers is high enough, leading high frequency signals on graph could not pass through filter, and then resulting in an insufficient approximation of the real function and finally causing a defective performance in tasks. To solve this problem, we propose a novel model, called DoreBer, for document-level RE task, which can obtain a higher quality of graph representation. Specifically, DoreBer performs estimation of filter over the normalized Laplacian spectrum of a graph by leveraging an order-K Bernstein polynomial approximation, and designs its spectral property by setting the coefficients of the Bernstein basis. Therefore, DoreBer can alleviate the over-smoothing problem to enhance learning ability of model. In addition, DoreBer has a higher interpretability for learned parameters of graph filter. We evaluate DoreBer on the DocRED public document-level RE dataset. Online experimental results demonstrate that DoreBer achieves significant performance improvements (2.72 and 2.76 higher on Ign
$F_{1}$
and
$F_{1}$
respectively), over the previous state-of-the-art on sequence-based method baseline. DoreBer reveals the potential of BernNet method in document-level relation extraction tasks and sheds light on a path to learn potential representation in high-dimensional data.The source code of this paper can be obtained from
https://github.com/factor77/DoreBer/
.