Introducing Confidence as a Weight in Random Forest

Random forest is one of the most popular supervised learning methods which is a collection of multiple decision trees. It is computationally fast, easy to use and gives reasonable performances over diversified applications. In a random forest, outcomes from multiple trees are aggregated to make the final decision where all trees get the same importance. However, it is natural that the performance of all trees is not equal and thus calculating the majority vote from these trees may not always give the correct decision. To improve this situation, we propose Confidence weighted Random Forest (CwRF) which uses a confidence factor for each leaf node in every decision tree and use that confidence to take weighted majority vote for making the final decision. A rigorous experiment over twenty-five datasets illustrates that the proposed CwRF outperforms three other state-of-the-art methods and provides significantly better performance over nineteen datasets out of twenty-five datasets at best.