Martingale Stopping Rule Application in Bayesian A/B Testing With Batch Data

Continuous monitoring in A/B testing is widely used for selecting options, variants, or treatments in healthcare analytics. However, the appropriate stopping strategy for A/B testing in continuous monitoring remains unclear. This study proposes a Bayesian A/B testing method for continuous monitoring, incorporating a martingale stopping strategy. A uniform Beta distribution is used as the prior in the Bayesian A/B testing method. The posterior expected loss is employed as the criterion for choosing an option. When the posterior expected loss falls below a preset threshold, the result of the continuous monitoring is accepted for the A/B testing. With the proposed method, continuous monitoring of the null hypothesis (that the conversion rate of A is less than that of B) is managed using the posterior expected loss. The results obtained using the martingale stopping strategy enhance the interpretability of the A/B testing process. Finally, the effectiveness of the Bayesian A/B testing method is validated through numerical experiments on both simulated data and real-world applications.