Achieving magnitude order improvement in Porter stemmer algorithm over multi-core architecture

NLP search takes a long amount of time due to large size of corpus, besides there are too many hits at the server. At present, the strategy to deal with search engines is to have many thousands of servers in order to provide real-time searches. Fast alternatives are therefore sought. In this paper, we present a pioneering work in this direction by taking word stemming, a crucial aspect of search and indexing algorithms and showing how significant performance gain can be accomplished by employing multi-core architectures, which will serve the purpose of home computers in near future. We present our analysis of Porter's stemming algorithm on Cell Broadband Engine and describe the manner in which SIMD operations can be utilized to maximize performance. Our results show that cell processors provide performance gains of over 50 times over popular Intel processors and hence possess tremendous potential for NLP-IR applications.