Source Code Statement Classification using ANTLR and Random Forest

In software development, source code analysis is essential to maintain software. One of the metrics used to assess software quality is software complexity, which can be calculated by analyzing source code statements like declaration, expression, and control. However, analyzing source code statements using an existing popular technique, like NLP, requires another structured form of data, which can be obtained by converting the source code into a token stream using a programming language processor. This study aims to get the best source code statement classification from six observed models and two source code forms. The models are Decision Tree, Naïve Bayes, SVM, kNN, Rocchio Algorithm and Random Forest. The source code forms are the raw source code and token stream object. The dataset collection used 562 rows of source code statements from an open-source Java project in a public repository. Next, we use ANTLR to transform the raw source code and get the token stream object. Later, the dataset was modeled using TF-IDF as an NLP technique to get the source code features. Then, six machine learning models were built and evaluated as a comparison for the classification process. As a result, Random Forest became the best model with the highest accuracy values among other machine learning models. Moreover, the token stream became the best object over source code as a contribution from the ANTLR. The model successfully predicted source code statements with a 96.1% accuracy score using the Random Forest model and ANTLR tools. In advance, this multiclass classification was also evaluated and gave a result. No declaration statement was predicted as a control statement, and vice versa (100% accuracy), but some miss prediction was observed in declaration-expression and expression-control pairs. Nevertheless, the Random Forest model’s high accuracy and precision make it suitable for classifying source code statements.