A BERT-based Approach with Relation-aware Attention for Knowledge Base Question Answering

Knowledge Base Question Answering (KBQA), which uses the facts in the knowledge base (KB) to answer natural language questions, has received extensive attention in recent years. The existing works mainly focus on the modeling method and neglect the relations between questions and KB facts, which might restrict the further improvements of the performance. To address this problem, this paper proposes a BERT-based approach for single-relation question answering (SR-QA), which consists of two models, entity linking and relation detection. For entity linking, we adopt pre-trained BERT and a heuristic algorithm to reduce the noise in the candidate facts. For relation detection, the existing approaches usually model the question and the candidate fact respectively before calculate their semantic similarity, which might lose part of the original interaction information between them. To work around this problem, a BERT-based model with relation-aware attention is proposed. We construct the question-fact pair as the input of pre-trained BERT to preserves the original interactive information. To bridge the semantic gap between the questions and the KB facts, we also use a relation-aware attention network to enhance the representation of candidates. The experimental results show that our entity linking model achieves new state-of-the-art results and our complete approach also achieves state-of-the-art accuracy of 80.9% on the SimpleQuestions dataset.