Analyzing discourse structure using lexical cohesion: a connectionist tool

Discourse analysis has become a major focus of research from various disciplines, including computer science, linguistics, and psychology, in recent decades. The increasing recognition of discourse structure in the field of textual information retrieval makes the development of a computational method necessary. The article attempts to describe a quantitative system of discourse analysis based on the study of cohesion. What distinguishes it from previous studies is that attention is not primarily focused on itemizing cohesive features between lexical items but on observing how they combine to organize texts. We present a connectionist tool for selecting the most representative segments from a text on the basis of repeated lexical features. This follows the work on lexical cohesion which is identified to be one of the key factors in contributing to textual continuity. A methodology is developed for the production of readable summary of text which is capable of some degree of automation.