Frequent Pattern Mining (FPM) from a time series has received much attention. The problem has been studied under various matching relations, including, more recently, the order-preserving (OP) matching relation, in which two time series match if their elements share the same relative order (i.e., the same ranks). Thus, a frequent OP pattern captures a trend shared by sufficiently many parts of the input time series. Here, we propose exact, highly scalable algorithms for FPM in the OP setting. Our algorithms employ an OP suffix tree (OPST) as an index to store and query time series efficiently. Unfortunately, there are no practical algorithms for OPST construction. Thus, we first propose a novel and practical $\mathcal {O}(n\sigma \log \sigma)$-time and $\mathcal {O}(n)$-space algorithm for constructing the OPST of a length-$n$ time series over an alphabet of size $\sigma$. We also propose an alternative faster OPST construction algorithm running in $\mathcal {O}(n\log \sigma)$ time using $\mathcal {O}(n)$ space; this algorithm is mainly of theoretical interest. Then, we propose an exact $\mathcal {O}(n)$-time and $\mathcal {O}(n)$-space algorithm for mining all maximal frequent OP patterns, given an OPST. This significantly improves on the state of the art, which takes $\Omega (n^{3})$ time in the worst case. We also formalize the notion of closed frequent OP patterns and propose an exact $\mathcal {O}(n)$-time and $\mathcal {O}(n)$-space algorithm for mining all closed frequent OP patterns, given an OPST. In addition, we show how to utilize the OP matching relation to efficiently compare two time series based on their trends. We evaluated our algorithms extensively through experiments and case studies on real-world, multi-million-letter time series, demonstrating both their efficiency and effectiveness.
The mode of a collection of values (i.e., the most frequent value in the collection) is a key summary statistic. Finding the mode in a given range of an array of values is thus of great importance, and constructing a data structure to solve this problem is in fact the well-known Range Mode problem. In this work, we introduce the Subtree Mode (SM) problem, the analogous problem in a leaf-colored tree, where the task is to compute the most frequent color in the leaves of the subtree of a given node. SM is motivated by several applications in domains such as text analytics and biology, where the data are hierarchical and can thus be represented as a (leaf-colored) tree. Our central contribution is a time-optimal algorithm for SM that computes the answer for every node of an input N-node tree in O(N) time. We further show how our solution can be adapted for node-colored trees, or for computing the k most frequent colors, in the optimal O(N) time, for any given k=O(1). Moreover, we prove that a similarly fast solution for when the input is a sink-colored directed acyclic graph instead of a leaf-colored tree is highly unlikely. Our experiments on real datasets with trees of up to 7.3 billion nodes demonstrate that our algorithm is faster than baselines by at least one order of magnitude and much more space efficient. Last, we present case studies showing the effectiveness of our approach in pattern mining and sequence-to-database search applications.
Enumerating or counting combinatorial objects in graphs is a fundamental data mining task. We consider the problem of assessing the number of Eulerian trails in directed graphs, which is formalized as follows: Given a directed graph G = (V, E), with V = n nodes and E = m edges, and an integer z, assess whether the number #ET (G) of Eulerian trails of G is at least z. This problem underlies many applications in domains ranging from data privacy to computational biology, data compression, and transportation networks. Practitioners currently address this problem by applying the famous BEST theorem, which, in fact, counts #ET (G) instead of just assessing whether #ET (G) >= z. Unfortunately, this solution takes O(n(omega)) arithmetic operations, where o <2.373 denotes the matrix multiplication exponent. Since in most real-world graphs, the number m of edges is comparable to the number n of nodes, and z is moderate in practice, the algorithmic challenge is: Can we solve the problem faster for certain values of m and z? We want to design a combinatorial algorithm for assessing whether #ET (G) >= z, which does not resort to the BEST theorem and has a predictably bounded cost as a function of m and z. We address this challenge as follows. We first introduce a general algorithmic scheme for assessing (and enumerating) Eulerian trails. We then introduce a novel tree data structure to reduce the number of iterations in this general scheme. Finally, we complement the above with further combinatorial insight leading to an algorithm with a worst-case bound of O(m min{z, #ET (G)}) time. Our experiments using six benchmark datasets with multi-million edges from different domains show that our implementations are up to two orders of magnitude faster than the BEST theorem, perform much fewer than mz iterations and scale near-linearly with m in most cases. Our experiments further show that our implementations bring substantial efficiency benefits in a data privacy application which employs the BEST theorem for the assessment.
Given a string $P$ of length $m$, a longer string $T$ of length $n>m$, and two integers $l\geq 0$ and $r\geq 0$, the context of $P$ in $T$ is the set of all string pairs $(L,R)$, with $|L|=l$ and $|R|=r$, such that the string $LPR$ occurs in $T$. We introduce two problems related to the notion of context: (1) the Contextual Pattern Mining (CPM) problem, which given $T$, $(m,l,r)$, and an integer $τ>0$, asks for outputting the context of each substring $P$ of length $m$ of $T$, provided that the size of the context of $P$ is at least $τ$; and (2) the Contextual Pattern Counting (CPC) problem, which asks for preprocessing $T$ so that the size of the context of a given query string $P$ of length $m$ can be found efficiently. For CPM, we propose a linear-work algorithm that either uses only internal memory, or a bounded amount of internal memory and external memory, which allows much larger datasets to be handled. For CPC, we propose an $\widetilde{\mathcal{O}}(n)$-space index that can be constructed in $\widetilde{\mathcal{O}}n)$ time and answers queries in $\mathcal{O}(m)+\widetilde{\mathcal{O}}(1)$ time. We further improve the practical performance of the CPC index by optimizations that exploit the LZ77 factorization of $T$ and an upper bound on the query length. Using billion-letter datasets from different domains, we show that the external memory version of our CPM algorithm can deal with very large datasets using a small amount of internal memory while its runtime is comparable to that of the internal memory version. Interestingly, we also show that our optimized index for CPC outperforms an approach based on the state of the art for the reporting version of CPC [Navarro, SPIRE 2020] in terms of query time, index size, construction time, and construction space, often by more than an order of magnitude.
Sparse suffix sorting is the problem of sorting $b=o(n)$ suffixes of a string of length $n$. Efficient sparse suffix sorting algorithms have existed for more than a decade. Despite the multitude of works and their justified claims for applications in text indexing, the existing algorithms have not been employed by practitioners. Arguably this is because there are no simple, direct, and efficient algorithms for sparse suffix array construction. We provide two new algorithms for constructing the sparse suffix and LCP arrays that are simultaneously simple, direct, small, and fast. In particular, our algorithms are: simple in the sense that they can be implemented using only basic data structures; direct in the sense that the output arrays are not a byproduct of constructing the sparse suffix tree or an LCE data structure; fast in the sense that they run in $\mathcal{O}(n\log b)$ time, in the worst case, or in $\mathcal{O}(n)$ time, when the total number of suffixes with an LCP value greater than $2^{\lfloor \log \frac{n}{b} \rfloor + 1}-1$ is in $\mathcal{O}(b/\log b)$, matching the time of the optimal yet much more complicated algorithms [Gawrychowski and Kociumaka, SODA 2017; Birenzwige et al., SODA 2020]; and small in the sense that they can be implemented using only $8b+o(b)$ machine words. Our algorithms are simplified, yet non-trivial, space-efficient adaptations of the Monte Carlo algorithm by I et al. for constructing the sparse suffix tree in $\mathcal{O}(n\log b)$ time [STACS 2014]. We also provide proof-of-concept experiments to justify our claims on simplicity and efficiency.
Minimizer sampling is one of the most widely-used mechanisms for sampling strings [Schleimer et al., SIGMOD 2003; Roberts et al., Bioinformatics 2004]. Let S = S[1] ... S[n] be a string over a totally ordered alphabet Sigma. Further let w >= 2 and k >= 1 be two integers. The minimizer of S[i .. i + w + k-2] is the smallest position in [i, i + w-1] where the lexicographically smallest length-k substring of S[i .. i + w + k-2] starts. The set of minimizers over all i is an element of [1, n-w-k + 2] is the set .Mw,k(S) of the minimizers of S. We consider the following basic problem: Given S, w, and k, can we efficiently compute a total order on Sigma that minimizes |.Mw,k(S)|? We show that this is unlikely by proving that the problem is NP-hard for any w >= 2 and k >= 1. Our result provides theoretical justification as to why there exist no exact algorithms for minimizing the minimizer samples, while there exists a plethora of heuristics for the same purpose.
Let Substr k ( X ) denote the set of length- k substrings of a given string X for a given integer k > 0. We study the following basic string problem, called z -Shortest S k -Equivalent Strings : Given a set S k of n length- k strings and an integer z > 0, list z shortest distinct strings T 1 , . . . , T z such that Substr k ( T i ) = S k , for all i ∈ [1 , z ]. The z -Shortest S k -Equivalent Strings problem arises naturally as an encoding problem in many real-world applications; e.g., in data privacy, in data compression, and in bioinformatics. The 1 -Shortest S k -Equivalent Strings , referred to as Shortest S k -Equivalent String , asks for a shortest string X such that Substr k ( X ) = S k . Our main contributions are summarized below: Given a directed graph G ( V, E ), the Directed Chinese Postman (DCP) problem asks for a shortest closed walk that visits every edge of G at least once. DCP can be solved in ˜ O ( | E || V | ) time using an algorithm for min-cost flow. We show, via a non-trivial reduction, that if Shortest S k -Equivalent String over a binary alphabet has a near-linear-time solution then so does DCP. We show that the length of a shortest string output by Shortest S k -Equivalent String is in O ( k + n 2 ). We generalize this bound by showing that the total length of z shortest strings is in O ( zk + zn 2 + z 2 n ). We derive these upper bounds by showing (asymptotically tight) bounds on the total length of z shortest Eulerian walks in general directed graphs. We present an algorithm for solving z -Shortest S k -Equivalent Strings in O ( nk + n 2 log 2 n + zn 2 log n + | output | ) time. If z = 1, the time becomes O ( nk + n 2 log 2 n ) by the fact that the size of the input is Θ( nk ) and the size of the output is O ( k + n 2 ).
Differential privacy has become the de facto privacy standard, as it is widely employed by various organizations. However, only a few research works have focused on strings (e.g., [1, 2, 5]). This is surprising, given that strings are fundamental in modeling, for example, genomic sequences, mobility traces, or text logs. One of the key tasks in string database analysis is to count the occurrences of string fragments in the database: for instance, extracting frequent patterns or ??-grams, or publishing word frequency statistics. Most existing differentially private algorithms for string mining are largely heuristic and come with little or no worst-case error analysis, which is a major limitation given the fact that strings appear in key domains for decision making.
We introduce a weighted and unconstrained variant of the well-known minimum k union problem: Given a bipartite graph G(U,V,E) with weights for all nodes in V, find a set S subset of V such that the ratio between the total weight of the nodes in S and the number of their distinct adjacent nodes in U is maximized. Our problem, which we term Heavy Nodes in a Small Neighborhood (HNSN), finds applications in marketing, team formation, and money laundering detection. For example, in the latter application, S represents bank account holders who obtain illicit money from some peers of a criminal and route it through their accounts to a target account belonging to the criminal. We prove that HNSN can be solved exactly in polynomial time via linear programming. We also develop several algorithms offering different effectiveness/efficiency trade-offs: an exact algorithm, based on node contraction, graph decomposition, and linear programming, as well as three peeling algorithms. The first peeling algorithm is a near-linear time approximation algorithm with a tight approximation ratio, the second is an iterative algorithm that converges to an optimal solution in a very small number of iterations in practice, and the third is a near-linear time greedy heuristic. In addition, we formalize a money laundering scenario involving multiple target accounts and show how our algorithms can be extended to deal with it. Our experiments on real and synthetic datasets show that our algorithms find (near-)optimal solutions, outperforming a natural baseline, and that they can detect money laundering more effectively and efficiently than two state-of-the-art methods.
Strings (sequences of elements) are often disseminated to support applications, e.g., in bioinformatics, web analysis, and transportation. Unfortunately, this may expose sensitive patterns that model confidential knowledge. Concealing the occurrences of sensitive patterns in a string (e.g., by deleting some elements) while minimizing the associated quality loss has been the objective of several string sanitization methods. However, real-world attackers are likely to possess background knowledge about the string, e.g., an individual’s genome sequence is almost identical to a reference genome sequence. In addition, it is good security practice to assume that the attacker will know the algorithm that has been used to sanitize the string (Kerckhoffs’ principle). Yet, all existing methods fail to protect strings against such attackers, risking privacy breaches in critical applications. In our work, we consider for the first time how to defend against strategic attackers who possess such knowledge. To achieve this, we propose a novel framework to sanitize a string by probabilistically replacing carefully selected patterns. As part of this framework, we design three mathematical programming algorithms which compute the optimal replacement probabilities under different objectives and constraints, offering different privacy gain / quality loss tradeoffs. Our algorithms protect against strategic attackers using new concepts and measures, protect sensitive patterns of any length, and can construct one or more optimally sanitized strings that can be used in applications such as frequent pattern mining. Our experiments using five real-world datasets from different domains show that all our algorithms are substantially more effective than a natural baseline (e.g., they offer up to 2 times more privacy when they are configured to incur the same quality loss, and up to 3 times lower quality loss when they are configured to offer the same privacy). They also show that two “hybrid” algorithms that we propose, based on combining elements of the above algorithms, inherit the advantages of their constituent algorithms. These results, coupled with the generality of our approach, make our algorithms practical and beneficial for deployment.
Missing values arise routinely in real-world sequential (string) datasets due to: (1) imprecise data measurements; (2) flexible sequence modeling, such as binding profiles of molecular sequences; or (3) the existence of confidential information in a dataset which has been deleted deliberately for privacy protection. In order to analyze such datasets, it is often important to replace each missing value, with one or more valid letters, in an efficient and effective way. Here we formalize this task as a combinatorial optimization problem: the set of constraints includes the context of the missing value (i.e., its vicinity) as well as a finite set of user-defined forbidden patterns, modeling, for instance, implausible or confidential patterns; and the objective function seeks to minimize the number of new letters we introduce. Algorithmically, our problem translates to finding shortest paths in special graphs that contain forbidden edges representing the forbidden patterns. Our work makes the following contributions: (1) we design a linear-time algorithm to solve this problem for strings over constant-sized alphabets; (2) we show how our algorithm can be effortlessly applied to fully sanitize a private string in the presence of a set of fixed-length forbidden patterns [Bernardini et al. 2021a]; (3) we propose a methodology for sanitizing and clustering a collection of private strings that utilizes our algorithm and an effective and efficiently computable distance measure; and (4) we present extensive experimental results showing that our methodology can efficiently sanitize a collection of private strings while preserving clustering quality, outperforming the state of the art and baselines. To arrive at our theoretical results, we employ techniques from formal languages and combinatorial pattern matching.
Applications in domains ranging from bioinformatics to advertising feature strings that come with numerical scores (utilities). The utilities quantify the importance, interest, profit, or risk of the letters occurring at every position of a string. Motivated by the ever-increasing rate of generating such data, as well as by their importance in several domains, we introduce Useful String Indexing (USI), a natural generalization of the classic String Indexing problem. Given a string S (the text) of length n, USI asks for preprocessing S into a compact data structure supporting the following queries efficiently: given a shorter string P (the pattern), return the global utility U(P) of P in S, where U is a function that maps any string P to a utility score based on the utilities of the letters of every occurrence of P in S. Our work also makes the following contributions: (1) We propose a novel and efficient data structure for USI based on finding the top-K frequent substrings of S. (2) We propose a linear-space data structure that can be used to mine the top-K frequent substrings of S or to tune the parameters of the USI data structure. (3) We propose a novel space-efficient algorithm for estimating the set of the top-K frequent substrings of S, thus improving the construction space of the data structure for USI. (4) We show that popular space-efficient top-K frequent item mining strategies employed by state-of-the-art algorithms do not smoothly translate from items to substrings. (5) Using billion-letter datasets, we experimentally demonstrate that: (i) our top-K frequent substring mining algorithms are accurate and scalable, unlike two state-of-the-art methods; and (ii) our USI data structures are up to 15 times faster in querying than 4 nontrivial baselines while occupying the same space with them.
Frequent pattern mining is a flagship problem in data mining. In its most basic form, it asks for the set of substrings of a given string S of length n that occur at least t times in S, for some integer T is an element of [1, n]. We introduce a resilient version of this classic problem, which we term the (t, kappa)Resilient Pattern Mining (RPM) problem. Given a string S of length n and two integers t, k. [1, n], RPM asks for the set of substrings of S that occur at least t times in S, even when the letters at any k positions of S are substituted by other letters. Unlike frequent substrings, resilient ones account for the fact that changes to string S are often expensive to handle or are unknown. We make the following contributions. First, we present RPM-DP, a simple exact O(n(3) k log n)-time and O(n(2))space algorithm for RPM that is based on an existing dynamic programming algorithm. Second, we propose RPM-ESA, an exact O(n log n)-time and O(n)-space algorithm for RPM, which employs advanced data structures and combinatorial insights. Third, we conduct experiments on real large-scale datasets from different domains demonstrating that: (I) The notion of resilient substrings is useful in analyzing genomic data and fundamentally different from that of frequent substrings, as frequent substrings are often not resilient and thus do not remain frequent for long in versioned datasets; (II) RPM-ESA is several orders of magnitude faster and more space-efficient than RPM-DP; and (III) Clustering based on resilient substrings is effective.
In many real-world database systems, a large fraction of the data is represented by strings: sequences of letters over some alphabet. This is because strings can easily encode data arising from different sources. It is often crucial to represent such string datasets in a compact form but also to simultaneously enable fast pattern matching queries. This is the classic text indexing problem. The four absolute measures anyone should pay attention to when designing or implementing a text index are: (i) index space; (ii) query time; (iii) construction space; and (iv) construction time. Unfortunately, however, most (if not all) widely-used indexes (e.g., suffix tree, suffix array, or their compressed counterparts) are not optimized for all four measures simultaneously, as it is difficult to have the best of all four worlds. Here, we take an important step in this direction by showing that text indexing with sampling based on locally consistent anchors (lc-anchors) offers remarkably good performance in all four measures, when we have at hand a lower bound ℓ on the length of the queried patterns — which is arguably a quite reasonable assumption in practical applications. Specifically, we improve on a recently proposed index that is based on bidirectional string anchors (bd-anchors), a new type of lc-anchors, by: (i) introducing a randomized counterpart of bd-anchors which outperforms bd-anchors; (ii) designing an average-case linear-time algorithm to compute (the randomized) bd-anchors; and (iii) developing a semi-external-memory implementation and an internal-memory implementation to construct the index in small space using near-optimal work. Our index offers average-case guarantees. In our experiments using real (benchmark) datasets of sizes up to 10GB, we show that it compares favorably based on the four measures to all classic indexes: (compressed) suffix tree; (compressed) suffix array; and the FM-index. We also present a counterpart of our index with worst-case guarantees based on the lc-anchors notion of partitioning sets. To the best of our knowledge, this is the first index achieving the best of all worlds in the regime where we have at hand a lower bound ℓ on the length of the queried patterns.
Text indexing is a fundamental and well-studied problem. Classic solutions either replace the original text with a compressed representation, e.g., the FM-index and its variants, or keep it uncompressed but attach some redundancy - an index - to accelerate matching. The former solutions thus retain excellent compressed space, but are slow in practice. The latter approaches, like the suffix array, instead sacrifice space for speed. We show that efficient text indexing can be achieved using just a small extra space on top of the original text, provided that the query patterns are sufficiently long. More specifically, we develop a new indexing paradigm in which a sketch of a query pattern is first matched against a sketch of the text. Once candidate matches are retrieved, they are verified using the original text. This paradigm is thus universal in the sense that it allows us to use any solution to index the sketched text, like a suffix array, FM-index, or r-index. We explore both the theory and the practice of this universal framework. With an extensive experimental analysis, we show that, surprisingly, universal indexes can be constructed much faster than their unsketched counterparts and take a fraction of the space, as a direct consequence of (i) having a lower bound on the length of patterns and (ii) working in sketch space. Furthermore, these data structures have the potential of retaining or even improving query time, because matching against the sketched text is faster and verifying candidates can be theoretically done in constant time per occurrence (or, in practice, by short and cache-friendly scans of the text). Finally, we discuss some important applications of this novel indexing paradigm to computational biology. We hypothesize that such indexes will be particularly effective when the queries are sufficiently long, and so demonstrate applications in long-read mapping.
Strings in the real world are often encoded with some level of uncertainty. In the character-level uncertainty model, an uncertain string $X$ of length $n$ on an alphabet $\Sigma$ is a sequence of $n$ probability distributions over $\Sigma$. Given an uncertain string $X$ and a weight threshold $\frac{1}{z}\in(0,1]$, we say that pattern $P$ occurs in $X$ at position $i$, if the product of probabilities of the letters of $P$ at positions $i,\ldots,i+|P|-1$ is at least $\frac{1}{z}$. While indexing standard strings for online pattern searches can be performed in linear time and space, indexing uncertain strings is much more challenging. Specifically, the state-of-the-art index for uncertain strings has $\mathcal{O}(nz)$ size, requires $\mathcal{O}(nz)$ time and $\mathcal{O}(nz)$ space to be constructed, and answers pattern matching queries in the optimal $\mathcal{O}(m+|\text{Occ}|)$ time, where $m$ is the length of $P$ and $|\text{Occ}|$ is the total number of occurrences of $P$ in $X$. For large $n$ and (moderate) $z$ values, this index is completely impractical to construct, which outweighs the benefit of the supported optimal pattern matching queries. We were thus motivated to design a space-efficient index at the expense of slower yet competitive pattern matching queries. We propose an index of $\mathcal{O}(\frac{nz}{\ell}\log z)$ expected size, which can be constructed using $\mathcal{O}(\frac{nz}{\ell}\log z)$ expected space, and supports very fast pattern matching queries in expectation, for patterns of length $m\geq \ell$. We have implemented and evaluated several versions of our index. The best-performing version of our index is up to two orders of magnitude smaller than the state of the art in terms of both index size and construction space, while offering faster or very competitive query and construction times.
Time series are ubiquitous in domains ranging from medicine to marketing and finance. Frequent Pattern Mining (FPM) from a time series has thus received much attention. Recently, it has been studied under the order-preserving (OP) matching relation stating that a match occurs when two time series have the same relative order on their elements. Here, we propose exact, highly scalable algorithms for FPM in the OP setting. Our algorithms employ an OP suffix tree (OPST) as an index to store and query time series efficiently. Unfortunately, there are no practical algorithms for OPST construction. Thus, we first propose a novel and practical 𝒪(nσlogσ)-time and 𝒪(n)-space algorithm for constructing the OPST of a length-n time series over an alphabet of size σ. We also propose an alternative faster OPST construction algorithm running in 𝒪(nlogσ) time using 𝒪(n) space; this algorithm is mainly of theoretical interest. Then, we propose an exact 𝒪(n)-time and 𝒪(n)-space algorithm for mining all maximal frequent OP patterns, given an OPST. This significantly improves on the state of the art, which takes Ω(n^3) time in the worst case. We also formalize the notion of closed frequent OP patterns and propose an exact 𝒪(n)-time and 𝒪(n)-space algorithm for mining all closed frequent OP patterns, given an OPST. We conducted experiments using real-world, multi-million letter time series showing that our 𝒪(nσlogσ)-time OPST construction algorithm runs in 𝒪(n) time on these datasets despite the 𝒪(nσlogσ) bound; that our frequent pattern mining algorithms are up to orders of magnitude faster than the state of the art and natural Apriori-like baselines; and that OP pattern-based clustering is effective.
A string is often provided with numerical scores (utilities) which quantify the importance, interest, profit, or risk of the letters occurring at every position of the string. For example, every DNA fragment produced by modern sequencing machines comes with a confidence score per position. Motivated by the abundance of strings with utilities, we introduce Utility-oriented String Mining (USM), a natural generalization of the classic frequent substring mining problem. Given a string S of length n and a threshold V, USM asks for every string R whose utility U (R) is at least V, where U is a function that maps R to a utility score based on the utilities of all letters of every occurrence of R in S. In addition, our work makes the following contributions: (1) We identify a class U of utility functions for which USM admits an 0(n2)-time algorithm. (2) We prove that no listing algorithm solves the USM problem in subquadratic time for every utility function, or even for every function in U. (3) We propose an O(n log n)-time algorithm that solves USM for a class of monotone functions from U. (4) We design another O(n log n)-time algorithm for the same problem that is comparable in runtime but offers drastic space savings in practice when, in addition, a lower bound on the length of the output strings is provided as input. (5) We demonstrate experimentally using publicly available, billion-letter datasets that our algorithms are many times more efficient, in terms of runtime and/or space, compared to an Apriori-like baseline which employs advanced string processing tools.
In the Pattern Masking for Dictionary Matching (PMDM) problem, we are given a dictionary $\mathcal{D}$ of $d$ strings, each of length $\ell$, a query string $q$ of length $\ell$, and a positive integer $z$, and we are asked to compute a smallest set $K\subseteq\{1,\ldots,\ell\}$, so that if $q[i]$, for all $i\in K$, is replaced by a wildcard, then $q$ matches at least $z$ strings from $\mathcal{D}$. The PMDM problem lies at the heart of two important applications featured in large-scale real-world systems: record linkage of databases that contain sensitive information, and query term dropping. In both applications, solving PMDM allows for providing data utility guarantees as opposed to existing approaches. We first show, through a reduction from the well-known $k$-Clique problem, that a decision version of the PMDM problem is NP-complete, even for strings over a binary alphabet. We present a data structure for PMDM that answers queries over $\mathcal{D}$ in time $\mathcal{O}(2^{\ell/2}(2^{\ell/2}+\tau)\ell)$ and requires space $\mathcal{O}(2^{\ell}d^2/\tau^2+2^{\ell/2}d)$, for any parameter $\tau\in[1,d]$. We also approach the problem from a more practical perspective. We show an $\mathcal{O}((d\ell)^{k/3}+d\ell)$-time and $\mathcal{O}(d\ell)$-space algorithm for PMDM if $k=|K|=\mathcal{O}(1)$. We generalize our exact algorithm to mask multiple query strings simultaneously. We complement our results by showing a two-way polynomial-time reduction between PMDM and the Minimum Union problem [Chlamtac et al., SODA 2017]. This gives a polynomial-time $\mathcal{O}(d^{1/4+\epsilon})$-approximation algorithm for PMDM, which is tight under plausible complexity conjectures.
Gabriele Fici合作论文数Universita di Palermo3