Model for Bahasa Text Cleaning Based Regular Expression Non-Deterministic Finite Automata Approach

This study emphasizes the importance of text cleaning as a crucial initial step in text processing and data mining, significantly impacting the quality of analysis and final results. Effective text cleaning improves data quality by removing noise, correcting formatting issues, and eliminating irrelevant words. This approach also plays a vital role in data preprocessing, enhancing contextual integrity, computational efficiency, reducing bias, and facilitating result interpretation. The study further explores the use of string matching algorithms with the Non-Deterministic Finite Automata (NFA) Equivalence approach and a-Move techniques in pattern matching within regular expressions, designed to address various challenges in text cleaning such as removing excess spaces and correcting character duplication. The testing results using the Token Similarity approach with cosine similarity show values predominantly very close to 1, indicating a high degree of similarity between token vectors in the analyzed documents. These values range from 0.859 to 1.000, reflecting minor variations in similarity. Values near 1 suggest strong semantic or contextual alignment, while lower values highlight less similarity. The consistent high values confirm the effectiveness of the cosine similarity method in capturing nuanced similarities within the data. Detailed results can be seen in Figures 2 and 3.