Pre-Trained Encoder Decoder Transformer Model for Language Processing

Tamil language processing in NLP has yet to be outstanding, mainly because of the absence of high-quality resources. In this project, a novel approach to address these limitations is to build an encoder-decoder transformer model specifically for the Tamil language. The comprehensive Tamil text dataset is augmented along with transformer architectures to further enhance the performance on tasks such as machine translation, text summarization, and generation. Character-level processing further allows Tamil to some extent to capture subtleties, while the multi-head self-attention mechanisms bring in a new dimension to unveil complex linguistic dependencies of the text. Preliminary results look promising for gaining this efficiency in language processing as it outperformed prevailing approaches in key Tamil NLP tasks. Using the Transformer model for processing the Tamil language holds potential for filling an important gap for underrepresented languages in NLP, and further contributes to developing more inclusive AI.