Embedding Meets Frequency: Novel Approaches to Stopword Identification in Burmese

In this study, we present a comprehensive exploration of stopword extraction techniques tailored for the Burmese language. Utilizing a manually segmented corpus of 212,836 Burmese sentences, we benchmark traditional methods such as term frequency and entropy against our novel proposals, word2vec_frequency and fasttext_frequency. Our findings showcase that these innovative embedding-frequency approaches not only align with the established behavior of Zipf’s law but also offer promising avenues for enhancing text analysis in Burmese and potentially other under-resourced languages.