A Random Forest Using a Multi-valued Decision Diagram on an FPGA

A random forest (RF) is a kind of an ensemblemachine learning algorithm used for a classification and aregression. It consists of multiple decision trees that are built fromrandomly sampled data. The RF has a simple, fast learning, andidentification capability compared with other machine learningalgorithms. It is widely used for various recognition systems. Theconventional RF consisted of binary decision trees (BDTs), whilein this paper, we used a multi-valued decision diagrams (MDDs). In the MDD, each variable appears only once on a path, however, in the BDT, some variable may appear multiple times. Sincethe path length is short in the MDD, it can be evaluated at ahigh speed. The disadvantage is that the number of nodes inthe MDD increases with O(2N), where N denotes the numberof input variables. Fortunately, random forests encourage to usethe small number of N for each tree in order to avoid overfitting. Therefore, in several data sets used in the experimental, the number of nodes did not increase even if the MDD wasused. To reduce the development time, the Altera SDK forOpenCL (AOCL), a kind of a high-level synthesis tool, was used. To accelerate the RF classification using the AOCL, we proposethe fully pipelined architecture to increase the memory bandwidthusing on-chip memories on the FPGA. Also, we apply optimalprecision fixed point representation instead of 32 bit floating pointone. We compared the performance with the CPU and the GPUimplementations. As for the LPS (lookups per second), the FPGArealization was 10.7 times faster than the GPU one, and it was14.0 times faster than the CPU one. As for the LPS per powerconsumption, the FPGA realization was 61.3 times better thanthe GPU one, and it was 12.1 times better than the CPU one.