Analyzing software reviews for software quality-based ranking

Software comparison reviews from users who have experience with the software are important information for software ranking. Software reviews may contain information in terms of software quality comparison such as, “Software A is better than the other in quality Q”. This information can be used for software quality ranking. This research presents an approach for software ranking from user reviews which composed of 3 main phases: user reviews gathering, user reviews analysis and software ranking. After user reviews gathering, a collection of user review is constructed and analyzed. The analysis consists of comparative relation extraction, quality classification and sentiment classification. We use keyword and grammatical relation to extract comparative relation. Keyword is also used for 5 types of quality classification; performance, reliability, security, usability and maintainability. Information from comparative relation is used for sentiment classification. Then, the ranking method is performed. From our experiment, we achieve a high value of f-measure for comparative relation extraction and quality classification. We also improve the accuracy of sentiment classification. Our proposed ranking result is statistically correlated to the expert judgment with 0.935 Pearson's correlation coefficient.