A Comparative Study on R Packages for Text Mining

The term
Text Mining
, which is given to the set of techniques used for the extraction, cleaning and processing of the information in texts, has become useful to provide valuable information to other algorithms and widely used with statistical and machine learning methods. By enabling the extraction of useful insights from textual data, Text Mining has become a potent tool in decision-making and knowledge discovery across many areas, including health care, government, education and industry. R is a mature open-source programming environment that has overstepped its initial scope of application for statistical computing and graphics to be used in pretty all the Data Science knowledge Area Groups. The objective of this paper is to present review and benchmarking analysis of packages for text mining techniques with
R
in computational systems. The paper reviews thirteen different packages comparing them on their execution time and memory used, for which new tests have been specifically designed. The results of this approach have been intended to be used over the most common tasks carried out when analyzing texts, and comparisons included allow R users to know which packages are best for each task and to improve their performance. Text mining package (
tm
) stands out particularly in Tokenization and Stemming techniques, while
fastTextR
is the best choice for Topic Modeling and Normalization. Also in the case of the Term Frequency-Inverse Document Frequency (TF-IDF) technique, the
textir
package is a clear choice. The other packages will depend on whether the technique is applied to a document-term matrix (DTM) or to plain text. In addition, there are packages that perform better in runtime than in memory usage and vice versa, making the choice more difficult. Packages such as
udpipe
can achieve better results working in parallel. Future works will include the same analysis for parallel computing, hybrid approaches, and novel algorithms.