A Malay text summarizer using pattern-growth method with sentence compression rules

A text summary is a condensed representation of text where salient information is extracted with the purpose to ease users' readability. However, if the summary was extracted “verbatim” from its source, the sentence may contain inessential information along with salient information that may effect on the overall coherence in the summary generation. The purpose of Sentence Compression in Text Summarization is to produce a compact and informational content by eliminating unnecessary constituent in a sentence. We introduce a Pattern-Growth text representation model named Frequent Adjacent Sequential Pattern (FASP) and Frequent Eliminated Pattern (FASPe) to represent the text using a set of sequence adjacent words or “textual pattern” that are frequently used and eliminated across the Malay news document collection. From the discovered textual pattern, we derived some heuristic Sentence Compression Rules in generating compressed sentences to construct a single extract Malay summary. We conducted experiments on Malay news dataset and the result demonstrates that moderate compressed summary using Sentence Compression Rules has better agreement with human composed summary.