Analyzing the properties of smoothing methods for language models

The authors discuss the properties of several frequent smoothing methods for language models. Because of the data sparseness problem, smoothing methods are employed to estimate the probability for each n-gram (including all the seen and unseen events). There are several well-known smoothing methods: Additive discount, Good-Turing, Witten-Bell, Katz and Absolute discount. We propose a set of properties to evaluate the statistical behaviors of these methods. Furthermore, a smoothing scheme, Huang-Yu method, is presented, which complies with all the proposed properties.