Advancing Named Entity Recognition with Gazetteer and Transformer-based language model for Hindi

This paper presents a technique for Hindi Named Entity Recognition (NER) using transformer encoders and a gazetteer-based approach. NER is a crucial task in natural language processing that involves identifying entities and their types in text sequences. However, real-world NER tasks are challenging due to short contexts and complex and emerging entities. To address these challenges, this technique uses a gazetteer-adapted integration network that combines the advantages of existing knowledge (gazetteer) with transformer-based encoders for predicting NER tags. However, integrating a gazetteer into the learning process can lead to weaker learning outcomes because the gazetteer contains specific entity information that can bias the learning process. To overcome this, the technique integrates the gazetteer and language model in a two-step process to minimize data loss and increase model efficiency. The technique uses MuRIL and XLM-R transformer-based language models for feature extraction. Additionally, the technique uses the SPARQL query language to extract entities from Wikidata to construct a Hindi language gazetteer for targeted entity types. The technique utilizes a Contextual Gazetteer Representation (GCR) encoder, which works in conjunction with a word-level model and a Mixture-of-Experts gating network. This method selectively combines context and gazetteer features based on certain conditions to address the issue of overuse of features. The experimental results show an improvement in performance, even with a lower coverage rate.