Tree-KGQA: An Unsupervised Approach for Question Answering Over Knowledge Graphs

Most Knowledge Graph-based Question Answering (KGQA) systems rely on training data to reach their optimal performance. However, acquiring training data for supervised systems is both time-consuming and resource-intensive. To address this, in this paper, we propose Tree-KGQA, an unsupervised KGQA system leveraging pre-trained language models and tree-based algorithms. Entity and relation linking are essential components of any KGQA system. We employ several pre-trained language models in the entity linking task to recognize the entities mentioned in the question and obtain the contextual representation for indexing. Furthermore, for relation linking we incorporate a pre-trained language model previously trained for language inference task. Finally, we introduce a novel algorithm for extracting the answer entities from a KG, where we construct a forest of interpretations and introduce tree-walking and tree disambiguation techniques. Our algorithm uses the linked relation and predicts the tree branches that eventually lead to the potential answer entities. The proposed method achieves 4.5% and 7.1% gains in F1 score in entity linking tasks on LC-QuAD 2.0 and LC-QuAD 2.0 (KBpearl) datasets, respectively, and a 5.4% increase in the relation linking task on LC-QuAD 2.0 (KBpearl). The comprehensive evaluations demonstrate that our unsupervised KGQA approach outperforms other supervised state-of-the-art methods on the WebQSP-WD test set (1.4% increase in F1 score) - without training on the target dataset.


I. INTRODUCTION
A knowledge graph can be viewed as an abstraction of the real world that describes real-world entities and their relationships. Knowledge graphs are widely used as a source of structured data for KG-based question answering, dialogue systems, retrieval systems. Since the advent of large-scale knowledge graphs (KG) such as DBpedia [1], Freebase [2], and Wikidata [3], KG-based systems have evolved significantly. Given a natural language question, the task of a KG-based question answering (KGQA) system is to retrieve the correct answer from the knowledge graph. Entity and The associate editor coordinating the review of this manuscript and approving it for publication was Gianmaria Silvello . relation linking are the primary sub-tasks of KGQA. These sub-tasks include determining the surface form (mentions in the question) of the entity and relation in the question and subsequently mapping them to the respective entity and relation in the knowledge graph. The linked entity and relation are then utilized to obtain the answer entity in the final step [4].
KGQA on both simple and complex questions is a well-researched topic [5]- [7]. For training, supervised systems depend heavily on knowledge graph-based question answering datasets. Reaching peak performance often requires a significant amount of training data [8], [9]. Since both data collection and training processes are time consuming and cost-intensive, this is a bottleneck in developing dataset-independent KGQA systems. Furthermore, VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ supervised systems are often vulnerable to brittleness [10].
Since they aim to capture the underlying dynamics in the training data, they frequently fail to generalize well when tested on previously unseen data. The KGQA task is depicted in Figure 1, where the circular nodes indicate entities and the connecting directed lines represent the relationship between two KG entities. To alleviate the time and effort necessary to develop a question answering (QA) system, researchers recently explored unsupervised and few-shot question answering techniques [11], [12]. Effective unsupervised KGQA is still a challenging research problem. Unsupervised KGQA is particularly hard because, firstly, large-scale knowledge graphs such as Wikidata [3] contain more than 80 million entities and a few thousand relations. Linking the entity and relation mentioned in the question to the corresponding large-scale KG entity and relation is thus a challenging task. Secondly, it is a standard practice to execute a query (e.g., using SPARQL) over the KG to extract answer entities [4], [13]. Query construction for this purpose adds an additional layer of difficulty.
Addressing the issues mentioned above, we propose a simple yet effective unsupervised KGQA method leveraging pre-trained language models. The primary motivation of this research is to develop a dataset-independent KGQA system, which can answer natural questions from various datasets without additional training or fine-tuning. We adopt powerful off-the-shelf language models pre-trained on named entity recognition (NER) and natural language inference tasks for the KGQA sub-tasks [14], [15]. Specifically, we split the KGQA task into three sub-tasks: entity linking, relation linking, and answer entity extraction. Firstly, we employ a BERT-based [14] pre-trained NER model to detect the surface form of the entity. Additionally, we pre-process and index the contextualized representation of the entities into a dense space for effective and fast candidate entity generation during the inference. The index is utilized to generate a set of candidate entities, which are then disambiguated to obtain the final predicted entity (details in Section III-A). Secondly, by combining the 1-hop connected relations of the entities linked in the previous step, a set of candidate relations for relation linking is created. A pre-trained BART model [15] is then applied to the candidate relations to obtain the most probable relation in a zero-shot manner (details in Section III-B). Finally, we construct a set of k-level trees from the k-hop sub-graphs of the linked entities. Then, tree-walking and treedisambiguation techniques are employed to extract answer entities from the constructed trees (details in Section III-C).
To assess the performance of our proposed approaches, we conduct experiments on four publicly available benchmarks: LC-QuAD 2.0 [16], LC-QuAD 2.0 (KBpearl) [17], QALD-7-Wiki [18], and WebQSP-WD [19]. The empirical study confirms that our proposed system achieves a significant improvement in entity and relation linking sub-tasks. In the entity linking task, we notice an absolute increase of 4.5% on the LC-QuAD 2.0, 7.1% on the LC-QuAD 2.0 (KBpearl), and 0.1% on the QALD-7-Wiki in F1 score. The improvement in relation linking is 5.4% on the LC-QuAD 2.0 (KBpearl) in F1 score. Despite the simplicity, our proposed Tree-KGQA achieves an absolute increase of 1.4% in the F1 score over the state-of-the-art methods without training on WebQSP-WD test set. To encourage further research on unsupervised KGQA, we have made our code open source. 1 We anticipate that our findings will lay the groundwork for further study on unsupervised KGQA. The contributions of this paper can be summarized as follows: • We propose an unsupervised entity linking method that achieves state-of-the-art (SOTA) results on LC-QuAD 2.0, LC-QuAD 2.0 (KBpearl), and QALD-7-Wiki datasets.
• We introduce a zero-shot relation linking mechanism that achieves SOTA results on the LC-QuAD 2.0 (KBpearl).
• We introduce a novel tree-walking and treedisambiguation techniques for extracting answer entities. In particular, we propose a modular and unsupervised KGQA system that does not require any training and can be applied to any Wikidata-based KGQA dataset. Finally, we establish a new baseline for KGQA on the LC-QuAD 2.0 KBpearl dataset.
Rest of the part of this paper is organised as follows. In Section II, we review the previous research efforts on various methods for entity linking, relation linking, and answer extraction. In Section III, we describe our proposed unsupervised KGQA approach which includes, unsupervised entity linking, unsupervised relation linking, and tree-walking based answer extraction method. In Section IV, we describe the experiments and results. A comprehensive analysis of the proposed system and its components is provided in Section V. Finally, in Section VI, we summarize the key findings and identify future study areas.

II. RELATED WORK
Our research mainly focuses on leveraging pre-trained language models for question answering over knowledge graphs (KGQA). The KGQA task is often divided into three atomic sub-tasks namely, entity linking, relation linking and answer entity extraction.

A. ENTITY LINKING
Previous works on entity linking primarily focused on detecting entity mentions in the question and then linking these mentions to the correct entity in the knowledge using entity labels as well as other features such as entity type information [20], [21]. Several studies in a separate line of research focused on training entity mention detection and entity disambiguation together to perform entity linking [8], [9]. However, in order to train these systems, it is necessary to have datasets with annotated entity mention boundaries. Recently, natural language processing has reached a new height of success with the emergence of Transformer-based [22] pretrained language models [14], [15]. In the context of question answering, pre-trained language models have been widely studied for the entity linking task [9], [23].

B. RELATION LINKING
Relation linking is another challenging task in KGQA since it requires complex language inference capabilities. Both supervised and distantly supervised approaches have been explored for the relation linking task [21], [24]. In a different research, systems use already linked entities from the preceding step to perform relation linking, utilizing the structural information of the knowledge graph [25]. Unseen relation linking has also been studied recently, where the model needs to predict relations which are not seen during the training step [26]. In a similar line of research [27], [28], models jointly use knowledge graph embedding for entity linking, where the linked relation information is used additionally to perform disambiguation among the candidate entities. In a disparate research, a zero-shot methodology has also been used to investigate relation linking [29].

C. ANSWER ENTITY EXTRACTION
The two most prevalent methodologies for the answer entity extraction sub-task are semantic parsing-based and retrievalbased methods. Semantic parsing-based methods transform the natural question into a logical form which is then utilized to fetch the answer entities from the target KG [30], [31]. On the contrary, retrieval-based methods use the entity and relation extracted from the natural question to obtain the answer entities from the KG [32], [33]. In a different line of research, a graph neural network-based method for KGQA has been proposed by Sorokin and Gurevych [19], while other approaches fetch candidate SPARQL queries using the entities and predicted relations and re-rank them using neural network-based methods [4], [34]. More recently, a messagepassing based system for the KGQA task has been developed, where a confidence score is propagated throughout the knowledge graph, computed by input question parsing and matching [5].
Several studies proposed pre-trained language modelbased zero-shot QA systems [35], [36]. In contrast to the previous works, our proposed system focuses on solving the KGQA problem in an unsupervised way, utilizing pre-trained language models without fine-tuning for entity and relation linking, and tree-based techniques for answer entity extraction.

III. APPROACH: TREE-KGQA
In this section, first, we define the knowledge graph and knowledge tree. Following that, we discuss each component of our proposed Tree-KGQA system in depth.

Definition 1 (Knowledge Graph):
A knowledge graph G, is a labelled and directed multi-graph consisting of a set of entities E as nodes and a set of relations R as edges between them. A k-hop sub-graph G k i associated to a node E i ∈ E, denotes the set of all the connected nodes and edges within the radius-k distance from node E i .

Definition 2 (Knowledge Tree):
A knowledge tree with klevels T k i , associated to an entity E i , is a labelled and directed tree; consisting of nodes and branches , where { , } ∈ G k i . A Forest F, is denoted as the set of knowledge trees; Given a natural language question Q, our proposed system aims to predict a set of answer entities E a ⊆ E that answers the question. Table 1 provides an overview of the notations of the concepts covered in this research.

A. ENTITY LINKING
The entity linking task entails a) mention detection -spotting the surface form of the entity that appears in the question and b) mapping the detected mention to the corresponding VOLUME 10, 2022  Figure b) demonstrates the candidate entity generation procedure given a detected entity mention. Sentence-BERT is used to obtain the vector representation of the entity mention lionel. The encoded vector is then passed to the FAISS module that performs a lookup into the dense space and generates N candidate entities that are similar to the provided entity span, lionel. The red circle represents the given entity mention in the dense space, where the other circles inside the larger orange circle indicate similar entities around it. knowledge graph entity. The steps involved in entity linking are described below.

1) MENTION DETECTION
To detect the entity mentions in the question, we employ a BERT-large [14] model pre-trained for the named entity recognition task.
The function f (·) in Equation 1, is a pre-trained BERTlarge model that takes a question Q as input and predicts a set of named entity word tokens, W m as the output. For instance, consider the question, Which football club does lionel play for?. The system detects lionel as the entity mention in this step using Equation 1. In the following steps, the detected entity mention is mapped or in other words linked to the corresponding knowledge graph entity.

2) ENTITY MAPPING
We first index all the entity labels from a target KG into a dense space as a pre-processing step of entity mapping. During inference, the system generates candidate entities from the dense space for each detected entity mention from the previous step. To obtain the final linked entity from the set of candidate entities, an additional entity disambiguation step is performed in the cases where the same entity label appears more than once. The entity mapping technique is explained in detail below.

a: ENTITY INDEXING
In this step, firstly, we extract all the entities from the target KG, in our case Wikidata, and store it in an Entity store (see Figure 2a). The Entity store contains all the Wikidata entity labels (e.g., Lionel Messi) and their Wikidata ID (e.g., Q615). Secondly, we encode all the knowledge graph entity labels using Sentence-BERT [37]. Sentence-BERT captures the overall meaning of the entity label better since entity labels frequently contain multiple words in them. We obtain a vector of dimension 1 × 768 for each entity label from Sentence-BERT. Finally, the encoded vector representations of the KG entities are indexed into a dense space using FAISS [38]. During the inference, the system utilises a hierarchical indexing algorithm IndexHNSWFlat from FAISS, which enables the system to generate candidate entities (see Figure 2b) in an optimized way [8], [9]. Given an entity span, the hierarchical indexing algorithm generates N candidate entities from the dense space based on k-nearest neighbors (KNN) approximate search.
For each detected entity span m i ∈ W m , the system performs entity linking separately. The system generates a set of N = 10 candidate entities E c i = {E 1 , E 2 , . . . , E N } for each entity mention m i ∈ W m , using FAISS (Figure 2b). Each generated candidate entity has an indexing score (from the FAISS approximate search) indicating how similar they are to the entity mention in the dense space. The candidate entity with the highest indexing score is then considered as the linked entity. Henceforth, a disambiguation step between the generated entity candidates is not required if all the candidate entity labels appeared once in the set.

b: ENTITY DISAMBIGUATION
The system performs entity disambiguation if an entity label appears multiple times in the candidate entity set. In that  Figure a) depicts a k-level tree (with k=2). Since a tree has many nodes and branches (edges), we present a toy example. Figure b) shows a forest consists of a set of trees constructed from the sub-graph of the linked entities. For the demonstration purpose, we show a forest consists of two trees. The red branches show the position of predicted relation in different trees. The green nodes represent the leaf nodes at level-k, where the blue nodes refer to the intermediary nodes between the root and leaf nodes. Furthermore, the yellow nodes represent the predicted answer entity nodes connected by the red branches.
case, it firstly predicts a temporary relation R t using Algorithm 5. Although we develop Algorithm 5 to perform relation linking (details in Section III-B), in this section we utilize Algorithm 5 to obtain R t . The question Q, and a set of all the 1-hop connected relations of the candidate entities are used as input to the Algorithm 5. As the output, Algorithm 5 predicts a relation which we denote as R t in this section. The system selects an entity with the highest similarity score from E c i as linked entity E m i , which is connected to the predicted relation R t at a distance of 1-hop in the KG. For instance, for the question Which company's CEO is Tim Cook?, the predicted entity mention is Tim Cook. The entity label Tim Cook appears multiple times in the set of generated candidate entities; hence, entity disambiguation is required. By utilizing Algorithm 5, CEO is obtained as R t . In the generate candidate entity set, Tim Cook (Q265852) has the relation CEO in its 1-hop connected relations. Where the other candidate entities with the same entity label (e.g., Tim Cook (Q7803347) an Australian rules footballer, Tim Cook (Q1404825) an American ice hockey player) do not have the relation CEO in their 1-hop connections. Consequently, Tim Cook (Q265852), an American business executive, gets predicted as the final linked entity. In the cases where there exist multiple candidate entities with the same label, and R t in their 1-hop, the entity with the highest indexing score that contains R t in its 1-hop is selected as the linked entity.
Finally, after repeating the whole entity mapping process for each entity mention, the system produces the final set of linked entities, E L as follows: For the running example question, the entity mention lionel gets linked to the Wikidata entity, Lionel Messi (Q615).

B. ZERO-SHOT RELATION LINKING
We model the relation linking problem as a classification task, where the system aims to link the given natural language question to one of the KG relations based on label information. In our proposed approach, we firstly generate a set of candidate relations R c from all the 1-hop connected relations of the already linked entities E L as follows: where h i denotes the set of 1-hop connected relations of the entity E i . For the running example question and linked entity Lionel Messi, the set of candidate relations R c is {citizen of, lives in, plays for}(see Figure 3a). Furthermore, we mask all the detected entity mentions in the question with a generic token <ENT>, to obtain a masked question representation denoted byQ, Which football club does <ENT> play for?. We mask the entity mentions in the question to reduce noises in the relation classification task. In Algorithm 5, the function maskEnt(.) masks the entities in the question. The system then performs zero-shot relation label classification, leveraging a pre-trained language model called BART [15], which was pre-trained for the natural language inference (NLI) task. In Equation 4, function Z(·) is a BARTlarge model [15] that computes the probability of being the correct relation label given the modified question (Q) and a set of candidate relation labels (labels of relations in R c ).
Here, r i ∈ R c is a candidate relation. Finally, we obtain the predicted relation R L as follows: From Equation 5, the system obtains plays for as the predicted and linked relation R L . Algorithm 5 summarizes the relation linking task described in this section.

C. ANSWER ENTITY EXTRACTION
To extract the answer entities from the knowledge graph, firstly, we build a forest utilizing the sub-graph information associated to the linked entities (obtained from Section III-A). Then, we perform tree-walking over all the trees within the constructed forest, using the relation predicted in Section III-B. Finally, we obtain the answer entities VOLUME 10, 2022

Algorithm 1: Relation Linking
Input: A question Q, a set of candidate relations R cand Output: where r i ∈ R cand 5 return R p from the tree, based on the tree-disambiguation technique following Algorithm 20.

1) BUILDING A FOREST
In order to build a forest, first we construct a set of knowledge-trees. For each linked entity E i ∈ E L , we generate a k-level tree T k i constructed from the k-hop sub-graph associated to E i as follows: The linked entity is designated as the tree's root node (in orange color) at level 0 ( Figure 3a). In this case, Lionel Messi is the root node of a tree. The other nodes and edges in the k-hop sub-graph of the linked entity are connected to the tree's root node at the same stage as they are in the sub-graph G k i . The function buildTree(·) in Algorithm 20, performs the tree-construction operation. A set of generated k-level trees are denoted as a forest F (as specified by the definition 2). In cases where no entities are linked, as predicted answer entities the system returns an empty set. For the running example question, the system constructs a forest with one tree for the linked entity Lionel Messi (Q615).
Each branch of the tree represents a relation between the parent and the child entity node. For instance in Figure 3a, a branch ''capital city'' connects a parent entity node, ''Spain'' and a child entity node, ''Madrid'' (Spain capital city − −−−− −→ Madrid). Each node in a tree preserves a state variable V, which holds a set of values {S r , K, and R max }. Where K denotes the tree level, R max the relation for which the node obtained the maximum score, and S r the maximum similarity score for the relation R max . During the answer entity extraction process, the values of the state variable aid in the treedisambiguation process. At this stage, all state variables are initialized with null value.

2) TREE-WALKING
In this step, the predicted relation R L performs tree-walking across all the trees in the forest, starting from the root node till the nodes at level-k of each tree. During the walk, for each tree T k i ∈ F the system computes embedding-based cosine similarity between the predicted relation R L and all the 1-hop connected branches h i of each node E i ∈ T k i . At each step of the walk, the system updates the node state

Algorithm 2: Answer Entity Extraction
Input: A forest F, predicted relation R pred and hops k Output: (value of S r and R max ) with the similarity scores of the connected 1-hop relations. The values of a node state only get updated when a higher value than the existing S r of that node is obtained for any connected relation (or branch). The function updateState(·) in Algorithm 20, updates the node state values with the values passed in as parameters. We employ QuatE [39], a knowledge graph embedding model trained on Wikidata, to compute the similarities between two relations in order to consider KG structural information during the process. In Algorithm 20, the function emb(·) takes a relation as input and returns the knowledge graph embedding of the relation from QuatE. Finally, the system selects all entities connected to the node with the highest S r value, by R max , as answer entities E a .

3) TREE-DISAMBIGUATION
We introduce a tree disambiguation technique for extracting the answer entities from the forest. In this technique, the system chooses the tree in which the node with the highest score (S r ) resides. If multiple trees have a node with the same maximum score in their node state, the tree with the highest scoring node at the lowest level (lower value of k) is chosen (Figure 3). Moreover, in rare cases (less than 1% in the WebQSP-WD dataset), when several trees have nodes

A. DATA
We chose Wikidata [3] (based on May 2019 English Wikipedia release) as the knowledge graph to gauge our proposed method since Wikidata is frequently used as a knowledge base for KGQA datasets. We evaluate our proposed method on four publicly available knowledge graph based question answering datasets: • LC-QuAD 2.0 [16]: A large-scale dataset on Wikidata Knowledge Graph which was generated semiautomatically and consists of complex questions and their paraphrases.
• WebQSP-WD [19]: A Wikidata-based question answering dataset constructed from the original Freebase-based WebQSP dataset [40]. It is noteworthy that the system can be extended to different knowledge graphs with low effort (discussed in Section V-D). Table 2 lists the statistics of the datasets used in this research.

B. EXPERIMENTAL SETUP
We run our experiments on a system with 28 CPU cores, 12GB of GPU memory, and 256GB of RAM. A pretrained BERT-large [14] model with 340M parameters and BART-large model [15] with 406M parameters are used in this paper. We use macro-F1 score to evaluate the components of our system similar to other baseline models [17], [41].

C. BASELINES
We select a wide range of baseline models related to KGQA sub-tasks. The baseline models used in this paper are summarised below: DBpedia Spotlight: An open-source tool and a popular baseline for the entity linking task in TAC-KBP [42], [43].
TagMe: An entity linking tool that index Wikipedia pages and performs annotation on a given text [44].
EARL: Jointly performs entity and relation linking from the knowledge graph, by solving a Traveling Salesman Problem on the candidate nodes [46].
ReMatch: A part-of-speech and dependency parsing based relation linking tool for question answering [47].
Falcon: A tool that jointly performs entity and relation linking leveraging the concept of morphology and knowledge graph information [21].
VCG: A jointly optimized model for entity mention detection and disambiguation using contextual information [48].
KBPearl-NN: A neural network based end-to-end system that performs joint entity and relation linking [17].
PNEL: A pointer network based entity linking system [41]. Falcon 2.0: A morphology based entity and relation linking system [49].
STAGG: A semantic parsing approach for question answering over knowledge graph [50]. A reimplementation of STAGG from Sorokin and Gurevych [19] to facilitate the KGQA task, is used as a baseline in this work.
GGNN: Uses a complex semantic parser for performing question answering over knowledge bases [19].
The baseline scores in this paper are all reported from [17], [19], [41].  Table 3 shows the entity linking performance of the baseline models and our approach on LC-QuAD 2.0. All the results reported in this section are on the [0, 1] scale and test split of the datasets. From the results in Table 3, it is evident that our system achieves higher precision, recall and F1 scores as compared to the other baseline models.

1) ENTITY LINKING
We notice a substantial improvement (increment of 7.1%) on LC-QuAD 2.0 KBpearl in entity linking, see Table 4. We observed the majority of baseline systems have either low accuracy or recall scores. This is mostly due to the fact that the dataset is complex and often comprises many things. Our proposed entity linking mechanism achieved a balanced precision and recall score, resulting in a superior F1 score. The entity linking result on the small yet challenging dataset (QALD-7-Wiki) is reported in Table 5. Improved results across several datasets verify the effectiveness of our unsupervised entity linking approach.

2) RELATION LINKING
The relation linking performance of the baseline models and our proposed approach on LC-QuAD 2.0 (KBpearl) is reported in Table 6. The baseline scores are reported as in    Lin et al. [17]. Our proposed zero-shot relation label classification approach achieves an increased score of 5.4% over the previous state-of-the-art models.

3) KGQA
We report the KGQA score on WebQSP-WD dataset in Table 7. Our introduced Tree-KGQA system achieves an improved result (1.4% rise in F1 score) compared to the previous KGQA baselines. The KGQA scores reported in this paper are computed with k = 2. Furthermore, we provide a new baseline for the KGQA task on the LC-QuAD 2.0 KBpearl test set in Table 9. Moreover, we report the component-wise results of our proposed techniques on WebQSP-WD dataset in Table 8. The entries with the approach KGQA ER reflect the KGQA score given the ground truth values of EL and RL. We observe an improved KGQA score with k = 2 than k = 1. It is noteworthy that increasing the value of k increases the search space. Although our system performs remarkably on the EL and answer entity extraction tasks, it has a relatively poor KGQA score due to the low RL score. Nevertheless, relation linking (RL) is a challenging task that is still far from being solved.

A. ABLATION STUDY
We conduct an ablation study to investigate the effectiveness of major components of our proposed system. Table 10 demonstrates the improvement that each of the components brings to the overall performance of the system. A TF-IDF based entity linking approach exhibits a low F1 score of 0.599, where our proposed indexing mechanism based approach achieves significant gain in the performance (+6.2% using Fasttext and +2.1% using Sentence-BERT embedding). A relation-based entity disambiguation method further improved the result by 1.8%. Our proposed BART-based relation linking approach demonstrates a remarkable improvement (+9.1%) over the cosine similarity based relation linking method.
Furthermore, we assess the performance of the answer extraction component without our proposed tree disambiguation technique. We extract the entities directly connected to the linked entities by the predicted relation as answer entities which achieves a low KGQA F1 score of 0.243. Then, we employ the tree-walking and tree-disambiguation technique which improves the F1 score by 2.1%. Moreover, we utilized knowledge graph-based embedding during the answer entity extraction procedure to compute the similarity between the predicted relation and the branches of every node in a tree. This method allows the system to surpass Fasttext embedding based similarity calculation by 0.8%. Table 11 shows two cases from the entity linking, relation linking and KGQA tasks. The entity and relation linking cases are from LC-QuAD 2.0, where the KGQA cases are from WebQSP-WD.

1) ENTITY LINKING (EL)
Our proposed approach correctly detected and linked the entity in the first case, where Falcon 2.0 and PNEL failed to link the correct entity. This is a challenging case since it contains a long entity span. The underlined texts indicate the entity span in the question. In the second case, all the systems failed to detect country as the entity. Although mahomoud abbas is correctly detected as entity mention by Falcon 2.0 and PNEL, they linked the entity mention to the wrong KG entity Mahmoud Abbas (Q10515624), who is a footballer. On the contrary, with the help of entity disambiguation where relation information is used, our method correctly linked the mention mahomoud abbas to the correct KG entity Mahmoud Abbas (127998), who is the head of a state.

2) RELATION LINKING (RL)
The first case comprises administrative territorial entity (P150) and instance of (P31) as the ground truth relation. Since instance of (P31) does not appear explicitly in the question, it is difficult for the systems to predict it as a relation. In the second case, our proposed Algorithm 5 correctly predicted the relation medical condition (P1050). We adopt a BART-large model [15] in Algorithm 5, pre-trained on natural language inference task, which gives better inference VOLUME 10, 2022 capabilities in identifying the correct relation from a set of candidate relations.

3) KGQA
Our proposed unsupervised KGQA approach correctly extracted the answer entity in the first case. In the second case, Florida Gators football (Q5461394) is given as the ground truth which can be inferred by the relation member of sports team (P54) connected to the entity Tim Tebow (Q517467). However, our system extracted all the entities as the answer entities that are connected to Tim Tebow (Q517467) by the relation member of sports team (P54).

C. ERROR ANALYSIS AND LIMITATIONS
We conducted an error analysis to understand the cases where our system is not performing as expected. We observed that our proposed entity linker is unable to detect entities that are not named entities such as president (Q30461) and governor (Q132050), since it is using NER for detecting the entity mention(s). Here, Q30461 and Q132050 are Wikidata ID of the respective entities.
The most challenging aspect of KGQA is relation identification. Relations with similar labels exist in the Wikidata KG, which are difficult for systems to differentiate. For instance, the relations head of government (P6) and head of state (P35). This issue becomes more visible when we found that, F1 score on top-3 predicted relation is 49.39 and in top-10 it is 57.66. The relation accuracy results reported in Table 6 are based on the top-1 predicted results from the proposed zero-shot relation linker. Our system fails to predict relations requiring more complex reasoning capabilities, such as hierarchical relationships. For instance, for the question ''Give me cinematic technique that contains the word tilt in their name'', the correct relation that can be used to answer the question is Instance of (P31), which our system failed to capture. Furthermore, our proposed zero-shot relation linker can only predict one relation. Although this is a limitation of the system, questions generally contain one relation in the context of question answering.
Although our proposed answer extraction method is fairly straightforward, we observe that the KGQA model mainly suffers in the cases where no entities are predicted and the cases where a wrong relation is predicted. Similar to the relation linking, our system also fails to extract the correct answer entities for cases where comparative or logical reasoning is required to answer the questions (E.g., Is Lake Baikal bigger than the Great Bear Lake?).

D. DISCUSSION
The improved entity linking performance of our proposed model across all the benchmark datasets provides a solid foundation for the KGQA task. Despite the fact that our proposed relation linking approach outperforming previous methods in complex QA, it could benefit further from better logical inference capabilities. Furthermore, we designed our system in a modular way so that it can be easily extended and used across different KGQA sub-tasks. Within the scope of this paper, we explored Wikidata based datasets. However, from the description of our approaches, we can intuitively say that our system can be adapted for other knowledge graph based datasets. For that, first, the pre-processing step where entity indexing is performed needs to be executed. Then, we need to obtain the relation embedding from a knowledge graph embedding model to perform tree-walking (Section III-C). Our proposed KGQA system is runtime efficient. Several factors contributed to the fast runtime of our system. In entity linking, the FAISS indexing technique provides fast candidate generation (takes ∼0.04 seconds to generate 10 candidates per question). The performance of the entity linking baselines is shown in Figure 4 (baseline runtimes are reported from Banerjee et al. [41]). Furthermore, the relation linking component requires ∼0.09 seconds per question. Moreover, our proposed tree-based answer extraction process takes ∼0.39 seconds per question. Overall, the system takes ∼0.76 seconds per question to perform the entire KGQA task.

VI. CONCLUSION AND FUTURE WORK
We presented Tree-KGQA, an unsupervised technique to perform KGQA without any explicit training. Despite the simplicity, our proposed pre-trained language model-based, unsupervised method outperforms existing supervised systems by a fair margin in all the sub-tasks involved in KGQA. To substantiate our claim, we evaluate our proposed system across several benchmark datasets. Tree-KGQA achieves 4.5%, 7.1%, and 0.1% improvement in the entity linking task on LC-QuAD 2.0, LC-QuAD 2.0 (KBpearl), and QALD-7-Wiki datasets, respectively. Furthermore, it achieves a 5.4% gain in the relation linking task on LC-QuAD 2.0 (KBpearl) and 1.4% improvement in the KGQA task on the WebQSP-WD test set. Although our system proves to be useful for the majority of the types of questions found in the datasets studied, further work is required to tackle more challenging questions requiring counting, comparisons, and logical reasoning capabilities. In our future work, we plan to perform an extensive evaluation on datasets that are based on other knowledge graphs such as DBpedia [1] and Freebase [2]. Additionally, we want to explore the possibility of advanced clustering methods such as [53], [54] for the entity clustering task.
MD RASHAD AL HASAN RONY received the bachelor's degree from BRAC University, Bangladesh, and the master's degree from the University of Bonn, Germany, where he is currently pursuing the Ph.D. degree with the Smart Data Analysis Group. He is currently a Research Scientist at Fraunhofer IAIS, Dresden. His primary research interests include knowledge graph based dialogue systems, question answering systems, evaluation of generative systems, machine reading comprehension, and language models. DEBANJAN CHAUDHURI is currently pursuing the Ph.D. degree with the University of Bonn. His Ph.D. thesis titled ''Enriching Text-Based Human-Machine Interactions with Additional World Knowledge.'' He is an AI Expert with Uniper. He has several years of expertise solving business challenges with machine learning, especially deep learning. He is actively working on document understanding and processing using natural language processing and computer vision. His research interests include chatbots, question answering, optical character recognition, and image generation using generative adversarial networks.
RICARDO USBECK received the Ph.D. degree from the University of Leipzig, in 2017. He has been a Junior Professor (W1TTW2) of semantic systems with the University of Hamburg, since May 2021. His main research interests include circulate around knowledge-driven, semantic technologies, and methods to enable computers to understand and help humans.
JENS LEHMANN received the Ph.D. degree (summa cum laude) from the University of Leipzig and the joint master's degree in computer science from the Technical University of Dresden and the University of Bristol. He is currently the Head of the Smart Data Analysis Research Group, a Full Professor with the University of Bonn, and a Lead Scientist with Fraunhofer IAIS. He has authored more than 100 publications, which were cited more than 18 000 times and have won 12 international awards. His research interests include semantic web technologies, question answering, machine learning, and knowledge graph analysis. He contributed to various open-source projects such as DL-Learner, SANSA, LinkedGeoData, and DBpedia.