Associative memory neural networks for information retrieval of text word pairs

Natural language information processing remains a challenge in linguistics. Existing methods for retrieval of text often use stemming to retain common roots and base recall on these root words. This requires the removal of stop words, e.g., numbers, symbols, high frequency bid low semantic weight words, thereby precluding phrases using these words. In addition stemming is a morphologic approach that cannot readily process homonyms, synonyms and certain inflectional and derived forms. Machine learning approaches assign words to categories, but application to a large corpus remains in debate. For this study, we describe the application of the Cortronic theory and methods developed by Robert Hecht-Nielsen (2002) to information retrieval of text word pairs. Hecht-Nielsen's theories build on associative memory artificial neural networks (AMNNs) introduced by Steinbuch (1961) and extended by Willshaw et al. (1969) especially in regards sparseness. The AMNNs are used to process a large corpus without excluding stop words, and retain the joint probability of mutual occurrences that allows rapid retrieval of word pairs. This AMNN approach includes three key components: representation of arbitrary objects (words), learning and knowledge accumulation based on measurement of co-occurrence and use of all the learned knowledge to produce (predict) the missing word in a phrase.