An evaluation of Lucene for keywords search in large-scale short text storage

Some popular Internet applications such as instant message, blog, twitter and Google buzz generate huge data of short text. These data can then be summarized, mined, and queried by other applications. To this end, suitable storage design with outstanding performance must be offered to address the question of real-time full text indexing and searching. This paper studies Lucene indexing and searching performance for short-text. It gives a comparison test between Lucene and Oracle Text, and then presents performance factors of Lucene for short text search. The experimental results show that Lucene can meets the needs and is far superior to Oracle Text in this typical scenario.