Utilizing Long Distance Word Dependencies for Automatic Speech Recognition

Statistical language models have been widely used in natural language processing (NLP) applications. N-gram has long been proven as a useful words representation technique for language models. However, n-gram assumes that the probability of any word in a sequence of words depends only on the previous n-1 consecutive words. Therefore, investigating the performance of long distance dependencies (LDDs) is an important research area to consider the words' relationships beyond n-1 proceeding words. LDDs aims at finding the words co-occurrences while relaxing the consecutive constraint through a wider window rather than two or three previous words. That is, LDDs are a set of association rules that go beyond the scope of n-gram. One possible use of LDDs is for N-best hypotheses rescoring in automatic speech recognition (ASR) systems. In this paper, we used a textual part of a speech corpus that contains 6,145 short sentences. The experimental results show that the predictive Apriori data-mining algorithm is a suitable candidate to generate the frequently appeared LDDs that also contains consecutive and nonconsecutive words' relationships. The study also reveals that extracting LDDs is a computation expensive task that requires high performance computing (HPC) environment.