The Comparative Analysis of Smith-Waterman Algorithm with Jaro-Winkler Algorithm for the Detection of Duplicate Health Related Records

Duplicate detection is a process of identifying a pair of words that refers to the same real-word object. Generally, words consist of letters that have a syntax representation. In most cases, words, such as names, are incorrectly spelt during data entry and that creates duplicate data and if it is unresolved could lead to inconsistency of data. Fundamental algorithms that are applied in the design of duplicate detection systems includes Smith-Waterman and Jaro-Winkler algorithms. The study compares and analyses the application of Smith-Waterman algorithm and Jaro-Winkler algorithm to find duplicate words in large dataset such as health dataset. The basis for comparison is to find how accurate these algorithms are in detecting duplicate words in large health dataset. The contribution of this paper is the use of transitive and symmetry property on both Smith-Waterman and Jaro-Winkler algorithm when large dataset is involved in the duplicate detection processes.