A Three-Technique Pioneering Study for Unveiling Stopwords in Koshur Language

The Kashmiri language, known as ‘Koshur’, is written in three different scripts: Sharda, Nastaliq, and Kashmiri-Devanagari. In the field of Natural Language Processing (NLP), stopword removal is an essential preprocessing technique that helps achieve data uniformity. For the Koshur language, there is a lack of availability of stopwords list in any of the scripts. This study aims to bridge that gap by creating a comprehensive Koshur stopword list. The authors have conducted pioneering work in generating a Koshur stopword list comprising 279 words. To develop this list, three different techniques were used by authors first, the corpora of Koshur poetry, stories, magazines, and various online sources were used, resulting in the extraction of 110 stopwords. The authors then extracted a list of stopwords from three different languages Hindi, English, and Punjabi, merging of which after translation and removal of duplicates and words irrelevant to the Koshur language, contributed in 147 stopwords. The third technique used was the frequency count. A frequency count on 27 Koshur poems and 9 folk stories was done to validate the list with 89% accuracy and also contributed to adding 22 new stopwords to the previous list. Frequency count is a method that can count the occurrence of each word present in the document. The final list was then validated by three language experts with no changes suggested by them, resulting in 100% accuracy. This research marks the first of its kind in creating a Koshur stopword list in two scripts, Devanagari and Perso-Arabic, contributing to the advancement of NLP applications for the Koshur language.