We present new shared-memory parallel algorithms for the bi-core decomposition problem, which discovers dense subgraphs in bipartite graphs and is the bipartite analogue of the classic k-core decomposition problem. We develop a theoretically-efficient parallel bi-core decomposition algorithm that discovers a hierarchy by peeling vertices from the graph in parallel. Our algorithm improves the span (parallel running time) over the state-of-the-art parallel bi-core decomposition algorithm, while matching the state-of-the-art sequential algorithm in work. We additionally prove the bi-core decomposition problem to be P-complete, meaning that a polylogarithmic span solution is unlikely under standard assumptions. We also devise a theoretically-efficient parallel bi-core index structure to allow for fast parallel queries of vertices in given cores.Finally, we propose a novel practical optimization that prunes unnecessary computations, and we provide optimized parallel implementations of our bi-core decomposition algorithms that are scalable and fast. Using 30 cores with two-way hyper-threading, our implementation achieves up to a 4.9x speedup over the state-of-the-art parallel algorithm. Our parallel index structure can be constructed up to 27.7x faster than the state-of-the-art sequential counterpart. Due to the improved storage format of our index structure, our parallel queries are up to 116.3x faster than the state-of-the-art sequential queries.
We revisit the problem of building static hash tables on the GPU and present an efficient implementation of bucketed hash tables. By decoupling the probing scheme from the hash table in-memory representation, we offer an implementation where the number of probes and the bucket size are the only factors limiting performance. Our analysis sweeps through the hash table parameter space for two probing schemes: cuckoo and iceberg hashing. We show that a bucketed cuckoo hash table (BCHT) that uses three hash functions outperforms alternative methods that use iceberg hashing and a cuckoo hash table that uses a bucket size of one. At load factors as high as 0.99, BCHT enjoys an average probe count of 1.43 during insertion. Using three hash functions only, positive and negative queries require at most 1.39 and 2.8 average probes per key, respectively.
Similarity searching is a well-studied problem in trajectory processing, where given a query trajectory, one wants to report all trajectories in the database similar to the query. Several works have focused on developing near neighbor data structures for trajectories under various metrics, e.g., Hausdorff distance, Frechet distance, etc. However, all near-neighbor data structures for these metrics use space that is super-linear (and sometimes a high-degree polynomial) in the size of the trajectory database. With trajectory datasets getting larger everyday, these data structures may not fit in RAM, and are likely stored on disk. In order to avoid expensive disk-accesses for every query, a lightweight filter in RAM that reports YES when there is a trajectory in the database similar to the query, and NO otherwise, is highly desirable. Furthermore, A long-standing criticism of standard measures for trajectories is that they ignore the temporal aspect of time-stamped trajectories, which holds critical information. This work focuses on optimizing near neighbor data structures by addressing the two issues above. We present a data structure that preprocesses a database T of trajectories so that, given a query trajectory Q and a similarity factor 0 <= rho <= 1, it answers Yes or No to whether there is a trajectory in T that is rho-similar to Q. Similarity between two trajectories here is defined as the Hamming distance between their binary sketches obtained from certain 3D lifts of the trajectories, where the third dimension incorporates the temporal aspect. With respect to this similarity measure, the data structure has only false positives and no false negatives. We show via experiments that the data structure has fast construction and query time, and gives up to 95% space savings on the original datasets. Additionally, our experiments indicate low mismatch rates between our similarity measure and Hausdorff and SketchMin distances, showing that the new measure is meaningful. We conclude that our data structure can be used as part of a two-tiered system in conjunction with a near-neighbor data structure on disk.
Reconfigurable networks have traditionally suffered from long reconfiguration times due to hardware limitations. With the emergence of new technologies that can reconfigure on the order of nanoseconds, the Oblivious Reconfigurable Network (ORN) design paradigm has been proposed to take advantage of this new capability. Two of the most important performance metrics for network operators considering ORNs are latency and throughput, which are inherently opposed and thus lead to a tradeoff. Previously, we constructed two families of ORN designs, called EBS and VBS, and proved that together they achieve optimal maximum latency (up to a constant factor) for any given throughput value r. Unfortunately, both families are defined only for very restrictive network sizes, N. This poses a challenge for implementing networks based on these designs in practice, as it is unlikely that a given system will have a suitable size. In this work, we extend both EBS and VBS to any network size while minimizing the impact on maximum latency and throughput. We prove that the extended versions achieve optimal maximum latency for all sufficiently large N, except when the throughput r exactly equals the reciprocal of an even integer.
Network administrators are interested in measuring the distribution of delays (the time between a request and its response), by directly running succinct algorithms within high-speed network devices. Unfortunately, the considerable gap between the small available memory and the huge volume of arriving traffic makes it challenging to design an algorithm that accurately measures delays. Existing algorithms exhibit bias against samples with higher delays. We present fridges, a novel data structure that corrects for the survivorship bias due to hash collisions, producing unbiased estimates of the delay distribution. The key idea is to consider a sample that was lucky enough to survive many insertions into the data structure as a representative for other similar samples that did not survive. We also show how to combine results from multiple fridges, each optimized for a different range of delays, for further accuracy gains. Simulation experiments show our design outperforms prior work using naive hash-indexed arrays, achieving 2x-4x memory saving. We implement a prototype P4 program running on the Intel Tofino programmable switch, using only moderate hardware resources.
Network survivability has been recognized as an issue of a major importance in terms of security, stability and prosperity. This paper studies fundamental properties of HyperX, an emerging topology for connecting supercomputing and datacenter networks. We focus on the establishment of paths with guaranteed survivability, allowing path availability even upon a restricted number of node failures. We first examine the availability of disjoint paths connecting a pair of input nodes. Disjoint paths guarantee path existence even upon a bounded number of link failures. We explore the inherent tradeoff between allowing slightly longer paths and the ability to extend available sets of mutually disjoint paths. Second, we study the availability of paths in a HyperX topology that already observed link failures. Such failures can increase the length of available paths or even eliminate connectivity between network parts. Last, we provide an evaluation to illustrate the analytical path availability along with the potential impact of failures.
We study flow scheduling under node capacity constraints. We are given capacitated nodes and an online sequence of jobs, each with a release time and a demand to be routed between two nodes. A schedule specifies which jobs are routed in each step, guaranteeing that the total demand on a node in any step is at most its capacity. A key metric in this scenario is response time: the time between a job's release and its completion. Prior work shows no un-augmented algorithm is competitive for average response time, and that a constant factor competitive ratio is achievable with augmentation exceeding 2 (Dinitz-Moseley Infocom 2020). For maximum response time, the best known result is a 2-competitive algorithm with a augmentation 4 (Jahanjou et al SPAA 2020). We improve these bounds under various response time objectives. We show that, without resource augmentation, the best competitive ratio for maximum response time is $\Omega(n)$, where $n$ is the number of nodes. Our Proportional Allocation algorithm uses $(1+\varepsilon)$ resource augmentation to achieve a $(1/\varepsilon)$-competitive ratio in the setting with general demands and capacities, and splittable jobs. Our Batch Decomposition algorithm is $2$-competitive (resp., optimal) for maximum response time using resource augmentation 2 (resp., 4) in the setting with unit demands and capacities, and unsplittable jobs. We also derive bounds for the simultaneous approximation of average and maximum response time metrics.
Parallelism has become extremely popular over the past decade, and there have been a lot of new parallel algorithms and software. The randomized work-stealing (RWS) scheduler plays a crucial role in this ecosystem. In this paper, we study two important topics related to the randomized work-stealing scheduler. Our first contribution is a simplified, classroom-ready version of analysis for the RWS scheduler. The theoretical efficiency of the RWS scheduler has been analyzed for a variety of settings, but most of them are quite complicated. In this paper, we show a new analysis, which we believe is easy to understand, and can be especially useful in education. We avoid using the potential function in the analysis, and we assume a highly asynchronous setting, which is more realistic for today's parallel machines. Our second and main contribution is some new parallel cache complexity for algorithms using the RWS scheduler. Although the sequential I/O model has been well-studied over the past decades, so far very few results have extended it to the parallel setting. The parallel cache bounds of many existing algorithms are affected by a polynomial of the span, which causes a significant overhead for high-span algorithms. Our new analysis decouples the span from the analysis of the parallel cache complexity. This allows us to show new parallel cache bounds for a list of classic algorithms. Our results are only a polylogarithmic factor off the lower bounds, and significantly improve previous results.
Every processor with multiple cores sharing a cache needs to implement a cache-replacement algorithm. Previous work demonstrated that the competitive ratio of a large class of online algorithms, including Least-Recently-Used (LRU), grows with the length of the input. Furthermore, even offline algorithms like Furthest-In-Future, the optimal algorithm in single-core caching, cannot compete in the multicore setting. These negative results motivate a more in-depth comparison of multicore caching algorithms via alternative analysis measures. Specifically, the power of the adversary to adapt to online algorithms suggests the need for a direct comparison of online algorithms to each other. In this paper, we introduce cyclic analysis, a generalization of bijective analysis introduced by Angelopoulos and Schweitzer [JACM'13]. Cyclic analysis captures the advantages of bijective analysis while offering flexibility that makes it more useful for comparing algorithms for a variety online problems. In particular, we take the first steps beyond worst-case analysis for analysis of multicore caching algorithms. We use cyclic analysis to establish relationships between multicore caching algorithms, including the advantage of LRU over all other multicore caching algorithms in the presence of locality of reference.
In this paper, we study new batch-dynamic algorithms for $k$-clique counting, which are dynamic algorithms where the updates are batches of edge insertions and deletions. We study this problem in the parallel setting, where the goal is to obtain algorithms with low (poly-logarithmic) depth. Our first result is a new parallel batch-dynamic triangle counting algorithm with $O(\Delta\sqrt{\Delta+m})$ amortized work and $O(\log^* (\Delta+m))$ depth with high probability (w.h.p.), and $O(\Delta+m)$ space for a batch of $\Delta$ edge insertions or deletions. Our second result is a simple parallel batch-dynamic $k$-clique counting algorithm that uses a newly developed parallel $k$-clique counting algorithm to bootstrap itself, by enumerating smaller cliques, and intersecting them with the batch. Instantiating this idea gives a simple batch-dynamic algorithm running in $O(\Delta(m+\Delta)\alpha^{k-4})$ expected work and $O(\log^{k-2} n)$ depth w.h.p., all in $O(m + \Delta)$ space. Our third result is an algebraic algorithm based on parallel fast matrix multiplication. Assuming that a parallel fast matrix multiplication algorithm exists with parallel matrix multiplication constant $\omega_p$, the same algorithm solves dynamic $k$-clique counting with $O\left(\min\left(\Delta m^{\frac{(2k - 1)\omega_p}{3(\omega_p + 1)}}, (\Delta+m)^{\frac{2(k + 1)\omega_p}{3(\omega_p + 1)}}\right)\right)$ amortized work, $O(\log (\Delta+m))$ depth, and $O\left((\Delta+m)^{\frac{2(k + 1)\omega_p}{3(\omega_p + 1)}}\right)$ space.
This paper studies the design of self-adjusting datacenter networks whose physical topology dynamically adapts to the workload, in an online and demand-aware manner. We propose ReNet, a self-adjusting network which does not require any predictions about future demands and amortizes reconfigurations: it performs as good as a hypothetical static algorithm with perfect knowledge of the future demand. In particular, we show that for arbitrary sparse communication demands, ReNets achieve static optimality, a fundamental property of learning algorithms, and that route lengths in ReNets are proportional to existing lower bounds, which are known to relate to an entropy metric of the demand. ReNets provide additional desirable properties such as compact and local routing and flat addressing therefore ensuring scalability and further reducing the overhead of reconfiguration. To achieve these properties, ReNets combine multiple self-adjusting tree topologies which are optimized toward individual sources, called ego-trees in this paper.
This paper revisits the online balanced repartitioning problem (introduced by Avin et al. at DISC 2016) which asks for a scheduler that dynamically collocates frequently communicating nodes, in order to reduce communication costs while minimizing migrations in distributed systems. More specifically, communication requests arrive online and need to be served, either remotely across different servers at cost 1, or locally within a server at cost 0; before serving a request, the online scheduler can change the mapping of nodes to servers, i.e., migrate nodes, at cost alpha f per node move. Avin et al. presented a deterministic O (k log k)-competitive algorithm, Crep, which is optimal up to a logarithmic factor; however, their algorithm has the drawback that it relies on expensive repartitioning operations which result in a super-polynomial runtime. Our main contribution is a different deterministic algorithm pCrep which achieves the same competitive ratio, but runs in polynomial time. Our algorithm monitors the connectivity of communication requests over time, rather than the density as in prior work by Avin et al.; this enables the polynomial runtime. We analyze pCrep both analytically and empirically.
Automatic differentiation (AD) is a technique for computing the derivative of function F : R-n -> R-m defined by a computer program. Modern applications of AD, such as machine learning, typically use AD to facilitate gradient-based optimization of an objective function for which m << n (often m = 1). As a result, these applications typically use reverse (or adjoint) mode AD to compute the gradient of F efficiently, in time Theta (m.T-1(F)), where T-1 is the work (serial running time) of F. Although the serial running time of reverse-mode AD has a well known relationship to the total work of F, general-purpose reverse-mode AD has proven challenging to parallelize in a work-efficient and scalable fashion, as simple approaches tend to result in poor performance or scalability. This paper introduces PARAD, a work-efficient parallel algorithm for reverse-mode AD of determinacy-race-free recursive fork-join programs. We analyze the performance of PARAD using work/span analysis. Given a program F with work T-1(F) and span (critical-path length) T-infinity(F), PARAD performs reverse-mode AD of F in O(m.T-1(F)) work and O(logm + log(T-1(F))T-infinity(F)) span. To the best of our knowledge, PARAD is the first parallel algorithm for performing reverse-mode AD that is both provably work-efficient and has span within a polylogarithmic factor of the original program F. We implemented PARAD as an extension of Adept, a C++ library for performing reverse-mode AD for serial programs that is known for its efficiency. Our implementation supports the use of Cilk fork-join parallelism and requires no programmer annotations of parallel control flow. Instead, it uses compiler instrumentation to dynamically trace a program's series-parallel structure, which is used to automatically parallelize the gradient computation via reverse-mode AD. On eight machine-learning benchmarks, our implementation of PARAD achieves 1:5x geometric-mean multiplicative work overhead relative to the serial Adept tool, and 8:9x geometric-mean self-relative speedup on 18 cores.
Recent work has investigated adaptive filters, which are filters that change their internal representation in response to queries that yield false positives. These include: (1) strongly adaptive filters, which guarantee a false-positive probability of at most for any query regardless of the history of prior queries, i.e., against adaptive adversaries, (2) support-optimal filters, which guarantee an average false-positive probability of at most over sufficiently large query sequences, when the adversary is oblivious, (3) other adaptive filters that change their representation and empirically perform better, but do not come with any specific provable guarantees beyond static filters. In this article, we investigate the performance advantages that strongly adaptive filters offer on (non-adversarial) skewed query distributions, which are common in database applications. In our theoretical and experimental results, we model query distribution skewness with the Zipfian distribution with parameter. We consider two strongly adaptive filters: the broom filter and the telescoping adaptive filter (TAF). We also consider two adaptive (but not strongly adaptive) filters: the adaptive cuckoo filter (ACF), and a non-adaptive rank-and-select quotient filter augmented with a cache of recent false positives, which we call the cache-augmented filter (CAF). We prove upper bounds on the false-positive rates of the broom filter, the TAF, and the CAF as a function of the Zipfian parameter as the length of the query sequence tends to infinity. We provide an implementation of the broom filter, based on the (non-adaptive) rank-and-select quotient filter. We validate the above bounds experimentally on synthetic Zipfian query sequences on the broom filter, the TAF, and the CAF. Finally, we measure the observed false-positive rate of the broom filter, the TAF, the CAF, and the ACF on highly skewed real-world network trace data. We find that all adaptive filters achieved 1-2 orders of magnitude lower false-positive rates than non-adaptive filters. We further find that the broom filter and the TAF outperform the CAF only when the ratio of distinct negative queries to positive set size is high; otherwise, the CAF and the strongly adaptive filters yield similar false-positive rates.
We introduce the Read-Only Semi-External (ROSE) Model for the design and analysis of algorithms on large graphs. As in the well-studied semi-external model for graph algorithms, we assume that the vertices but not the edges fit in a small fast (shared) random-access memory, the edges reside in an unbounded (shared) external memory, and transfers between the two memories are done in blocks of size B. A key difference in ROSE, however, is that the external memory can be read from but not written to. This difference is motivated by important practical considerations: because the graph is not modified, a single instance of the graph can be shared among parallel processors and even among unrelated concurrent graph algorithms without synchronization, that instance can be stored compressed without the need for re-compression, the graph can be accessed without cache coherence issues, and the wear-out problems of non-volatile memory, such as Optane NVRAM, can be avoided. Using ROSE, we analyze parallel algorithms (some existing, some new) for 18 fundamental graph problems. We show that these algorithms are work-efficient, highly parallel, and read the external memory using only a block-friendly (and compression-friendly) primitive: fetch all the edges for a given vertex. Analyzing the maximum times this primitive is called for any vertex yields an (often tight) bound on the (low) I/O cost of our algorithms. We present new, speciallydesigned ROSE algorithms for triangle counting, FRT trees, and strongly connected components, devising new parallel algorithm techniques for ROSE and beyond.
Many parallel algorithms use at least linear auxiliary space in the size of the input to enable computations to be done independently without conflicts. Unfortunately, this extra space can be prohibitive for memory-limited machines, preventing large inputs from being processed. Therefore, it is desirable to design parallel in-place algorithms that use sublinear (or even polylogarithmic) auxiliary space. In this paper, we bridge the gap between theory and practice for parallel in-place (PIP) algorithms. We first define two computational models based on fork-join parallelism, which reflect modern parallel programming environments. We then introduce a variety of new parallel in-place algorithms that are simple and efficient, both in theory and in practice. Our algorithmic highlight is the Decomposable Property introduced in this paper, which enables existing non-in-place but highly-optimized parallel algorithms to be converted into parallel in-place algorithms. Using this property, we obtain algorithms for random permutation, list contraction, tree contraction, and merging that take linear work, $O(n^{1-\epsilon})$ auxiliary space, and $O(n^\epsilon\cdot\text{polylog}(n))$ span for $0<\epsilon<1$. We also present new parallel in-place algorithms for scan, filter, merge, connectivity, biconnectivity, and minimum spanning forest using other techniques. In addition to theoretical results, we present experimental results for implementations of many of our parallel in-place algorithms. We show that on a 72-core machine with two-way hyper-threading, the parallel in-place algorithms usually outperform existing parallel algorithms for the same problems that use linear auxiliary space, indicating that the theory developed in this paper indeed leads to practical benefits in terms of both space usage and running time.
In order to provide a high resilience and to react quickly to link failures, modern computer networks support fully decentralized flow rerouting, also known as local fast failover. In a nutshell, the task of a local fast failover algorithm is to pre-define fast failover rules for each node using locally available information only. These rules determine for each incoming link from which a packet may arrive and the set of local link failures (i.e., the failed links incident to a node), on which outgoing link a packet should be forwarded. Ideally, such a local fast failover algorithm provides a perfect resilience deterministically: a packet emitted from any source can reach any target, as long as the underlying network remains connected. Feigenbaum et al. (ACM PODC 2012) and also Chiesa et al. (IEEE/ACM Trans. Netw. 2017) showed that it is not always possible to provide perfect resilience. Interestingly, not much more is known currently about the feasibility of perfect resilience. This paper revisits perfect resilience with local fast failover, both in a model where the source can and cannot be used for forwarding decisions. We first derive several fairly general impossibility results: By establishing a connection between graph minors and resilience, we prove that it is impossible to achieve perfect resilience on any non-planar graph; furthermore, while planarity is necessary, it is also not sufficient for perfect resilience. On the positive side, we show that graph families closed under link subdivision allow for simple and efficient failover algorithms which simply skip failed links. We demonstrate this technique by deriving perfect resilience for outerplanar graphs and related scenarios, as well as for scenarios where the source and target are topologically close after failures.
We consider gradient descent like algorithms for Support Vector Machine (SVM) training when the data is in relational form. The gradient of the SVM objective can not be efficiently computed by known techniques as it suffers from the ``subtraction problem''. We first show that the subtraction problem can not be surmounted by showing that computing any constant approximation of the gradient of the SVM objective function is $\#P$-hard, even for acyclic joins. We, however, circumvent the subtraction problem by restricting our attention to stable instances, which intuitively are instances where a nearly optimal solution remains nearly optimal if the points are perturbed slightly. We give an efficient algorithm that computes a ``pseudo-gradient'' that guarantees convergence for stable instances at a rate comparable to that achieved by using the actual gradient. We believe that our results suggest that this sort of stability the analysis would likely yield useful insight in the context of designing algorithms on relational data for other learning problems in which the subtraction problem arises.
We consider the problem of evaluating certain types of functional aggregation queries on relational data subject to additive inequalities. Such aggregation queries, with a smallish number of additive inequalities, arise naturally/commonly in many applications, particularly in learning applications. We give a relatively complete categorization of the computational complexity of such problems. We first show that the problem is NP-hard, even in the case of one additive inequality. Thus we turn to approximating the query. Our main result is an efficient algorithm for approximating, with arbitrarily small relative error, many natural aggregation queries with one additive inequality. We give examples of natural queries that can be efficiently solved using this algorithm. In contrast, we show that the situation with two additive inequalities is quite different, by showing that it is NP-hard to evaluate simple aggregation queries, with two additive inequalities, with any bounded relative error.
