IndoJavaneseNLI: A Cross-Lingual Natural Language Inference Dataset for East Javanese “Ngoko” Register

Natural Language Inference (NLI) is a task that focuses on establishing the logical relationship between two sentences, premise sentences, and hypothesis sentences, by classifying it into “entailment”, “neutral”, and “contradiction”. The growth of NLI models has been noteworthy in the English language. However, little to no progress has been made on low-resource languages, such as Javanese. What’s more, the resulting translation from Machine Translation is blind to the cultural nuance of word choices. It tends to mix up words from various Javanese registers like “Ngoko”, “Madya”, and “Krama”. To tackle that issue, we present IndoJavaneseNLI, a cross-lingual NLI dataset for East Javanese “Ngoko” registers. Our dataset consists of the premise sentences in the Indonesian language and the hypothesis sentences in the Javanese language. This paper describes how we carefully inspect the Javanese translation, the process leading up to the building of good East Javanese “Ngoko” sentences, and its evaluation with Transformer-based models and transfer learning. We also found that XLMR yields the best result in solving NLI problems with our dataset with the accuracy of 67.56% on the fine-tuning method and 47.34% on the transfer learning method.