An Integer Programming-Based Approach to Construct Exact Two-Sample Binomial Tests with Maximum Power

Traditional hypothesis tests for differences between binomial proportions are at risk of being too liberal (Wald test) or overly conservative (Fisher's exact test). This problem is exacerbated in small samples. Regulators favour exact tests, which provide robust type I error control, even though they may have lower power than non-exact tests. To target an exact test with high power, we extend and evaluate an overlooked approach, proposed in 1969, which determines the rejection region through a binary decision for each outcome vector and uses integer programming to, in line with the Neyman-Pearson paradigm, find an optimal decision boundary that maximizes a power objective subject to type I error constraints. Despite only evaluating the type I error rate for a finite parameter set, our approach guarantees type I error control over the full parameter space. Our results show that the test maximizing average power exhibits remarkable robustness, often showing highest power among comparators while maintaining exact type I error control. The method can be further tailored to prior beliefs by using a weighted average. The findings highlight both the method's practical utility and how techniques from combinatorial optimization can improve statistical methodology.