Formatted Encryption Beyond Regular Languages

Format-preserving and format-transforming encryption (FPE and FTE, respectively) are relatively new cryptographic primitives, yet are already being used in a broad range of real-world applications. The most flexible existing FPE and FTE implementations use regular expressions to specify plaintext and/or ciphertext formats. These constructions rely on the ability to efficiently map strings accepted by a regular expression to integers and back, called ranking and unranking, respectively.In this paper, we provide new algorithms that allow encryption with formats specified by context-free grammars (CFGs). Our work allows for CFGs as they appear in practice, partly a pure grammar for describing syntax, and partly a set of lexical rules for handling tokens. We describe a new relaxed ranking method, structural ranking, that naturally accommodates practical CFGs, thereby empowering new FPE and FTE designs. We provide a new code library for implementing structural ranking, and a tool that turns a simple YACC/LEX-style grammar specification into ranking code. Our experimental analysis of the code shows that the new CFG ranking algorithm is efficient in interesting settings, even when the grammars are ambiguous. For example, we show that one can efficiently rank C programs of size thousands of kilobytes in milliseconds.