Dependency Parser Based Textual Entailment System

The development of a parser based textual entailment system that is based on comparing the dependency relations in both the text and the hypothesis has been reported. The textual entailment system uses the CCG Parser and the Stanford Parser. The Dependency Parser has been run on the 2-way Parser Training and Evaluation (PETE) (SemEval-2010 Evaluation Exercises on Semantic Evaluation Task 12 Parser Evaluation using Textual Entailment) trial set and the dependency relations obtained for a text and hypothesis pair has been compared. Some of the important comparisons are: subject-verb comparison, subject-subject comparison, object-verb comparison and cross subject-verb comparison. Each of the matches is assigned some weight learned from the PETE trial set corpus. A threshold has been set on the fraction of matching hypothesis relations for YES entailment decision based on the PETE trial set. The threshold score has been applied on the PETE test set using the same methods of dependency parsing followed by comparisons. Evaluation scores for Run 1 (CCG Parser output), obtained on the test set show 58.19% precision and 45.51% recall for YES decisions and 52.51% precision and 64.82% recall for NO decisions. Evaluation scores for Run 2 (Stanford Parser output), obtained on the test set show 55.68% precision and 59.61% recall for YES decisions and 52.23% precision and 48.61% recall for NO decisions. Evaluation scores for Run 3 (combining the output from CCG parser and Stanford Parser), obtained on the test set show 57.14% precision and 74.35% recall for YES decisions and 59.18% precision and 40% recall for NO decisions.