Punjabi Grammar Checker: Automatic Correction of Punjabi Language Sentences Using Transformer

Language is an immense basis of information transmission in social life. Grammar checking is the basic requirement for writing any documentation in particular language. Much research work has been done on grammar checker for English language. There has not been much research work done in other languages like Punjabi, not due to lack of interest but due to lack of resources. In Punjabi language, there is need of more attention. Advancement in deep neural networks in specific to natural language processing has encouraged researchers to apply these practices in resolving various challenging problems like automatic grammar checking. In this paper, we address the comparison and analysis of two deep neural network models Encoder-Decoder with attention mechanism and transformer for the problem of Punjabi sentence correction. We proposed neural machine translation approach for task of Punjabi Grammar Checker where input sentence is potentially erroneous and target sentence should be correct form of the input. In this paper the proposed approach is implemented with encoder-decoder using attention mechanism, base transformer with self-attention layers and fine-tuned transformer. The proposed approach with such models is validated on the Punjabi Grammar error dataset. Experiments show that the encoder-decoder model with attention mechanism has a good performance on smaller sequences but for longer sequences it has shown poor results but in contrast base transformer model has shown good and our proposed fine-tuned model has shown better performance to correct Punjabi sentences of small and longer sequences. Based on different samples of Punjabi dataset we have trained and tested such models and found our proposed approach with fine-tuned transformer model achieved 63% test accuracy on real test dataset which is better than other implemented models.