Keyword extraction using Renyi entropy: a statistical and domain independent method

Enormous data is currently available in textual form and is increasing at a very fast pace through various social, academic and economic activities. The need of the hour is to assimilate, analyse, interpret this unstructured data and put the inferences for better use wherever and whenever required. This is where keyword extraction plays an important role by helping in determining the relevant documents from a large pool of available data. In this article we propose and analyse a new domain independent statistical method for keyword extraction using Rényi entropy. Both actual and relative performance of proposed word ranking metric has been discussed. Results of experimental evaluation indicate that Rényi entropy-based word ranking metric has reliable performance and is coherent with previously defined entropy-based methods. Being a statistical method, it is computationally less intensive and domain independent method, and could be of great utility in organizing dynamic text collection and other applications.