Generation of Non-matching Strings for Testing Regular Expressions Via Complement Automaton Coverage.

Regular expressions (regexes) are widely used but error-prone, making effective testing critical. A common testing approach involves generating a set of strings from a given regex and checking whether they are accepted or rejected as intended. However, existing string generation methods focus mainly on matching strings, overlooking non-matching strings that are crucial for detecting overly restrictive errors, a type of fault that accounts for a majority of real-world regex bugs. In this paper, we propose a method for systematically generating non-matching strings to test regexes. We adapt graph-based edge-pair coverage to the complement automaton of the regex and perform a depth-first traversal to generate a set of non-matching strings satisfying this criterion. To keep test suites compact, we incorporate a three-level reduction strategy including alphabet-level reduction, path-level reduction, and semantic-level filtering. Experimental results on a dataset containing 1,738 faulty regexes detectable by non-matching strings show that our method achieves 68.53 × improvement over the nearest competitor. Furthermore, it yields compact test suites with an average of 24 strings per regex, where 90 ≤ 10 .