A framework for analyzing community detection algorithms

Evaluation of community detection algorithms is very important to ensure both accuracy and quality of identified communities. Measuring quality incorporates edges, while measuring accuracy involves node labels. Due to this fundamental difference between accuracy and quality, often the evaluation process confronts with the issues such as trade-off between the two. In addition, real world networks such as social networks are of unknown structure and complex. Accuracy of communities detected with any algorithm for these networks cannot be measured due to unavailability of ground truth. In such cases, the algorithms are certainly more likely to predict accurate communities that show higher inclination towards accuracy in the networks where ground truths are available. In this paper, a framework is proposed to analyze Relative Inclination Towards accuracy (RITA) of a set of community detection algorithms. The RITA analysis gives an intuition about how likely an algorithm would produce accurate communities in the networks where ground truth is not available. Moreover, the RITA analysis overcomes trade-off between accuracy and quality by incorporating both into a common platform. Results on variety of networks show the competency of proposed framework in dealing with the trade-off during analysis.