Uncertainty Quantification and Confidence Intervals for Naive Rare-Event Estimators

We consider the estimation of rare-event probabilities using sampleproportions output by naive Monte Carlo or collected data. Unlike usingvariance reduction techniques, this naive estimator does not have a priorirelative efficiency guarantee. On the other hand, due to the recent surge ofsophisticated rare-event problems arising in safety evaluations of intelligentsystems, efficiency-guaranteed variance reduction may face implementationchallenges which, coupled with the availability of computation or datacollection power, motivate the use of such a naive estimator. In this paper westudy the uncertainty quantification, namely the construction, coveragevalidity and tightness of confidence intervals, for rare-event probabilitiesusing only sample proportions. In addition to the known normality, Wilson's andexact intervals, we investigate and compare them with two new intervals derivedfrom Chernoff's inequality and the Berry-Esseen theorem. Moreover, wegeneralize our results to the natural situation where sampling stops byreaching a target number of rare-event hits. Our findings show that thenormality and Wilson's intervals are not always valid, but they are close tothe newly developed valid intervals in terms of half-width. In contrast, theexact interval is conservative, but safely guarantees the attainment of thenominal confidence level. Our new intervals, while being more conservative thanthe exact interval, provide useful insights in understanding the tightness ofthe considered intervals.