Direct Estimation of Likelihood Ratio for the Analysis of Context

Direct estimation methods of likelihood ratios are reported to show better results than conventional estimation methods. Previous studies on direct estimation have usually dealt with continuous probability distributions; therefore, they cannot be applied to discrete distributions. Thus, we propose a direct estimation method for a discrete distribution, of which the choice of basis functions is the most important contribution. Our proposed basis functions are found to provide simple and exact solutions for the minimization of error; thus, we can directly estimate the likelihood ratios of bigrams that have a discrete distribution in a corpus. We verified the effectiveness of the proposed method by estimating the likelihood ratio of a bigram appearing in one apparent context over another. Experimental results show that our method is more accurate than conventional methods.