Grammar Inference of Mixed Notation Language Using ALBERT Model for String Validation

Grammar inference is a machine learning branch that can be applied in a variety of contexts. One popular approach for inferring grammars is using deep neural networks. Recurrent neural network (RNN) architecture and its derivatives have been used in the past for this task due to their sequential nature. However there have been limitations in the kinds of grammars they can learn such as context-free grammars. Meanwhile, the transformer architecture has demonstrated remarkable success in various language modeling tasks. For this reason, transformer-based network ALBERT, was fine-tuned for the classification of a simple context-free grammar which mixes infix and postfix arithmetic notation. The fine-tuned model displayed an accuracy of 0.956 and an F1 score of 0.955.