Bootstrap Pearson Test for Difference in Discrete Distributions

A one-dimensional discrete random variable X describes two populations. A random sample is drawn from each. We formalize a generic statistical test to determine whether the samples provide enough evidence that the distributions of X in the two populations are different. After showing the drawbacks of the existing analytical tests, we develop a Bootstrap procedure with Pearson test statistic calculated from a contingency table. The p-value is estimated using the simulated conditional distribution of the test statistics under null hypothesis for equality of population distributions. Compared to analytical tests, our procedure gives higher precision and decreases uncertainty in both small and big samples. We apply our procedure to survey data on leadership capabilities in academia among two samples of respondents. We take one question from the survey and analyse significance of differences in the responses depending on three criteria.