Measuring Lexical Cohesion in a Document

Lexical Cohesion is one of the important features in text processing for analyzing document structure and improving the accuracy of text processing. We present a hierarchial graph based model to measure cohesion by grouping lexically cohesive units together in a text. Latent semantic analysis is used to construct a relational graph to uncover the hidden semantics from the text by discovering new relations and their weights. Coherence score is derived from the learned weights that are assigned to edges in the graph and number of disjoint partitions in the graph.