Integrating Structural Priors into Transformer for Named Entity Recognition

Named entity recognition (NER) is a fundamental task in the natural language processing (NLP) field. Recently, owing to the superiority to parallelize and model long-term dependencies, Transformer has been widely used in various NLP tasks. However, such position-independent model is weak at capturing the relative position between sequences of words which leads to the performance drop when directly applied to NER. In this paper, we propose a straightforward yet effective multi-mask attention, aiming at integrating structural priors such as relative distance and dependency structure into Transformer to better model sequential order. Experiments on four NER benchmarks demonstrate the effectiveness of our method against several strong baselines.