Multi-Attention Mechanism and Dynamic Threshold Guided Multi-source Denoising for Distant Supervision Relation Extraction

Distant supervision reduces manual annotation costs in relationship extraction. However, existing research inadequately addresses multi-source noise, including word-level, sentence-level, and bag-level. This work introduces Multi-Attention and Dynamic Threshold (MADT) to mitigate these limitations. MADT employs a dynamic threshold method to eliminate noisy sentences and enhance positive example weights. A multi-level attention mechanism handles noise in words and bags, utilizing BERT and a bidirectional gated recurrent unit-self-attention mechanism network to extract entity and keyword features. An attention mechanism based on semantic similarity reduces the weight of noisy bags. Experimental results show that after three levels of noise reduction, the P@Average value reached 92.5% and 83.6% in NYT10d and NYT10m respectively, proving the effectiveness of noise reduction.