Flexibly Incorporating Prior Information into Subset Scan Statistics

We propose the matched filter subset scan (MFSS), a new approach for flexibly incorporating prior information into subset scanning while maintaining computational efficiency. MFSS encodes structured information about which subsets of the data are a priori more likely by (i) defining a set of subsets (the “filter set” Θ ) that collectively describe what we expect the true signal to look like; and (ii) optimizing a penalized log-likelihood ratio statistic, where the penalty is proportional to the Hamming distance to the nearest filter in Θ . We demonstrate that this approach maintains computational efficiency and exact detection of the highest-scoring subset for small-sized filter sets, and present initial results for scaling to larger-sized filter sets. Finally, we prove several theoretical results, including asymptotic identification of the correct filter and asymptotic improvement in detection accuracy, under reasonable simplifying assumptions.