Margin based permutation variable importance: A stable importance measure for random forest

Permutation based variable importance measure (VIM) has been widely used in various research fields. For example, in gene expression studies, it has been regarded as a screening tool to select a subset of relevant genes for subsequent analysis or better predictive performance. However, little effort has been devoted to the stability of variable importance measures. In this paper, margin based permutation variable importance measures (VIM-MDs) are proposed, which utilize the similarity between margin distribution before and after random permutation to evaluate the importance of variables. Experiments on six benchmark datasets show that the VIM-MDs outperform permutation based variable importance measure in terms of both global stability and predictive accuracy, which indicates that the proposed method could be used as an effective and stable variable importance measure for random forest.