This paper does not aim to prove new mathematical theorems or claim a fundamental unification of physics and information, but rather to provide a new pedagogical framework for interpreting foundational results in algorithmic information theory. Our focus is on understanding the profound connection between entropy and Kolmogorov complexity. We achieve this by applying these concepts to a physical model. Our work is centered on the distinction, first articulated by Boltzmann, between observable low-complexity macrostates and unobservable high-complexity microstates. We re-examine the known relationships linking complexity and probability, as detailed in works like Li and Vitányi’s An Introduction to Kolmogorov Complexity and Its Applications. Our contribution is to explicitly identify the abstract complexity of a probability distribution K(ρ) with the concrete physical complexity of a macrostate K(M). Using this framework, we explore the “Not Alone” principle, which states that a high-complexity microstate must belong to a large cluster of peers sharing the same simple properties. We show how this result is a natural consequence of our physical framework, thus providing a clear intuitive model for understanding how algorithmic information imposes structural constraints on physical systems. We end by exploring concrete properties in physics, resolving a few apparent paradoxes, and revealing how these laws are the statistical consequences of simple rules.

This paper presents a proof of the existence of computationally fast probabilistic erasure codes at distance ∈ from being MDS, namely the decoding algorithm is able with high probability to reconstruct the n letter message from any set of (1+∈)n letters. It can either be fixed rate or a rateless LT code [10] in that any number of code letters can be produced and each is produced independently of the others. We also decrease the minimum packet size from many to one letter. The key ingredient is a scheme Hierarchical Bun- dle/Bin (HB) which splits the message into a hierarchy of disjoint bundles and produces coded packets about each bundle. We show a correspondence of this to a particular game having to do with randomly throwing balls into a hierarchy of bins. The “information” that does not over flow from a smaller bin, contributes to the next larger bin that it is contained in. We prove matching upper and lower bounds on the cost of this game and provide the implementation details. This analysis is somewhat analogous to the evolution of the “ripple” in the LT decoding analysis [10]. The bundle size corresponds to the degree of the packet, therefore, smaller bundles tend to reduce encoding/decoding complexity, but packets coming from larger bundles ensure the approximately-MDS constraint, by ensuring more coverage. Our HB scheme with largest block size b requires encoding and decoding time O(∈b2) rather than the O(b2) needed for Reed-Solomon codes. This scheme HB (together with Spielman’s expanders) gives a probabilistic code with running time O(∈-1 ln(∈-1)n). Alon and Luby [5, 6] simultaneously developed a deterministic version but their running time is O(∈-4n). Both our and Alon’s results have since been completely subsumed by the latest generation of Shokrollahi and Luby’s Raptor codes [13, 14, 11].

In this work, we study time/space trade-offs for function composition. We prove asymptotically optimal lower bounds for function composition in the setting of nondeterministic read once branching programs, for the syntactic model as well as the stronger semantic model of read-once nondeterministic computation. We prove that such branching programs for solving the tree evaluation problem over an alphabet of size k requires size roughly kω(h), i.e space ω(h log k). Our lower bound nearly matches the natural upper bound which follows the best strategy for black-white pebbling the underlying tree. While previous super-polynomial lower bounds have been proven for read-once nondeterministic branching programs (for both the syntactic as well as the semantic models), we give the first lower bounds for iterated function composition, and in these models our lower bounds are near optimal.

We present a Polyhedral Scene Generator system which creates a random scene based on a few user parameters, renders the scene from random view points and creates a dataset containing the renderings and corresponding annotation files. We hope that this generator will enable research on how a program could parse a scene if it had multiple viewpoints to consider. For ambiguous scenes, typically people move their head or change their position to see the scene from different angles as well as seeing how it changes while they move; this research field is called active perception. The random scene generator presented is designed to support research in this field by generating images of scenes with known complexity characteristics and with verifiable properties with respect to the distribution of features across a population. Thus, it is well-suited for research in active perception without the requirement of a live 3D environment and mobile sensing agent, including comparative performance evaluations. The system is publicly available at https://polyhedral.eecs.yorku.ca.

We study the advice complexity of an online version of the set cover problem. The goal is to quantify the information that online algorithms for this problem need to be supplied with to compute high-quality solutions and to overcome the drawback of not knowing future requests. This concept was successfully applied to many prominent online problems in the past while trying to capture the essence of "what makes an online problem hard." The online set cover problem was introduced by Alon et al. (2009) [2]: for a ground set of size n and a set family of m subsets of the ground set, we obtain bounds in both n and m. We show that a linear number (with respect to both n and m) of advice bits is both sufficient and necessary to perform optimally. Furthermore, we prove that O((nlog⁡c)/c) bits are sufficient to design a c-competitive online algorithm, and this bound is tight up to a factor of O(log⁡c). We further give upper and lower bounds for achieving c-competitiveness with respect to m. Finally, we analyze the advice complexity of the problem with respect to some natural parameters, i.e., measurable properties of the inputs.

Efficient medium access control (MAC) is a key part of any wireless network communication architecture. MAC protocols are needed for nodes to access the shared wireless medium efficiently. Vehicular Ad hoc networks (VANETs) are an emerging network technology on the verge of large-scale deployment. The dynamic network topologies in VANETs caused by high mobility rates of vehicles presents a great challenge in reliable data transfer. A MAC protocol that enables quick reservation of packet transmission slots by vehicles that wish to send packets is crucial in addressing this challenge. In this paper, we propose a new distributed MAC algorithm ResVMAC for VANETs. We demonstrate using simulations, that our algorithm outperforms two state-of-the-art algorithms in key performance metrics.

Proving superpolylogarithmic lower bounds for dynamic data structures has remained an open problem despite years of research. Pǎtraşcu proposed an exciting approach for breaking this barrier via a two-player communication model in which one player gets private advice at the beginning of the protocol. He gave reductions from the problem of solving an asymmetric version of set-disjointness in his model to a diverse collection of natural dynamic data structure problems in the cell probe model. He also conjectured that, for any hard problem in the standard two-party communication model, the asymmetric version of the problem is hard in his model, provided not too much advice is given. In this paper, we prove several surprising results about his model. We show that there exist Boolean functions requiring linear randomized communication complexity in the two-party model, for which the asymmetric versions in his model have deterministic protocols with exponentially smaller complexity. For set-disjointness, which also requires linear randomized communication complexity in the two-party model, we give a deterministic protocol for the asymmetric version in his model with a quadratic improvement in complexity. These results demonstrate that Pǎtraşcu's conjecture, as stated, is false. In addition, we show that the randomized and deterministic communication complexities of problems in his model differ by no more than a logarithmic multiplicative factor. We also prove lower bounds in some restricted versions of this model for natural functions such as set-disjointness and inner product. All of our upper bounds conform to these restrictions. Moreover, a special case of one of these lower bounds implies a new proof of a strong lower bound on the tradeoff between the query time and the amortized update time of dynamic data structures with nonadaptive query algorithms.

Proving superpolylogarithmic lower bounds for dynamic data structures has remained an open problem despite years of research. Pǎtraşcu proposed an exciting approach for breaking this barrier via a two player communication model in which one player gets private advice at the beginning of the protocol. He gave reductions from the problem of solving an asymmetric version of set-disjointness in his model to a diverse collection of natural dynamic data structure problems in the cell probe model. He also conjectured that, for any hard problem in the standard two-party communication model, the asymmetric version of the problem is hard in his model, provided not too much advice is given. In this paper, we prove several surprising results about his model. We show that there exist Boolean functions requiring linear randomized communication complexity in the two-party model, for which the asymmetric versions in his model have deterministic protocols with exponentially smaller complexity. For set-disjointness, which also requires linear randomized communication complexity in the two-party model, we give a deterministic protocol for the asymmetric version in his model with a quadratic improvement in complexity. These results demonstrate that Pǎtraşcu’s conjecture, as stated, is false. In addition, we show that the randomized and deterministic communication complexities of problems in his model differ by no more than a logarithmic multiplicative factor. We also prove lower bounds in some restricted versions of this model for natural functions such as set-disjointness and inner product. All of our upper bounds conform to these restrictions. Moreover, a special case of one of these lower bounds implies a new proof of a strong lower bound on the tradeoff between the query time and the amortized update time of dynamic datastructures with non-adaptive query algorithms.

We prove exponential lower bounds on the size of semantic read-once 3-ary nondeterministic branching programs. Prior to our result the best that was known was for D-ary branching programs with |D| >= 2^{13}.

Efficient medium access control (MAC) algorithms are needed for nodes to share a transmission medium and achieve a high throughput. A MAC algorithm schedules packet transmissions so as to that minimize the time taken to send the packets without collisions. In wireless ad hoc and sensor networks, a MAC algorithm must conserve energy as well as provide good throughput. Most existing MAC algorithms for wireless networks are designed to work well under low traffic rates. In this paper we propose a new distributed algorithm Ad-ATMA for wireless ad hoc and sensor networks under relatively high traffic rates. We demonstrate using simulations that Ad-ATMA outperforms the best existing algorithms designed for higher traffic rates in terms of packet delivery ratio and latency while consuming almost identical energy as them.

John Tsotsos asked his AI students to work on having an AI solve a jigsaw puzzle, though it is NP-complete. This paper considers the search time of the recursive backtracking algorithm for solving it when the puzzle pieces are generated randomly with a planted solution. Feeling that a jigsaw puzzle should have a particular image when completed, we set the probability that two pieces fit together locally to be just small enough that with high probability the planted solution is unique. We were surprised to see that in this case the recursive backtracking algorithm which expands a rectangle from a corner has expected 1 + Θ(ǫ) branching width. In contrast, this width is exponential if it attempts to put together just the edge pieces into a frame, build a triangle from a corner, or build a block in the middle. If the probability of pieces fitting together is increased just to the point that there is an exponential number of complete solutions, then no matter which order the pieces are put together it takes an exponential amount of time to find one of these exponentially many solutions. It is also interesting that for the different orders of completing the puzzle, there are very different “reasons” for the exponential blowup. We coded the algorithm and it ran just as described.

John Tsotsos asked his AI students to work on having an AI solve a jigsaw puzzle, though it is NP-complete. Instead of looking at a real puzzle, this paper considers a randomly generated puzzle with a planted solution whose probability of piece connection is set so that each piece connects to Θ( √ ǫn) pieces but with 1 − Θ( ǫ √ n ) probability the planted solution is unique. As expected, the recursive backtracking algorithm, which attempts to put together just the edge pieces into a frame, branches exponentially as does building a triangle from a corner of the puzzle. Building a block in the middle does better, but still branches super (quasi) polynomially. In contrast, expanding a rectangle from a corner takes only linear expected time (or quadratic depending on the model). In fact, the expected number of backtracking branches alive at any given time is 1+Θ(ǫ). It is also interesting that for the different orders of completing the puzzle, there are very different “reasons” for the exponential blowup. We coded the algorithm and it ran just as described.

Proving superpolylogarithmic lower bounds for dynamic data structures has remained an open problem despite years of research. Recently, Patrascu proposed an exciting new approach for breaking this barrier via a two player communication model in which one player gets private advice at the beginning of the protocol. He gave reductions from the problem of solving an asymmetric version of set-disjointness in his model to a diverse collection of natural dynamic data structure problems in the cell probe model. He also conjectured that, for any hard problem in the standard two-party communication model, the asymmetric version of the problem is hard in his model, provided not too much advice is given.In this paper, we prove several surprising results about his model. We show that there exist Boolean functions requiring linear randomized communication complexity in the two-party model, for which the asymmetric versions in his model have deterministic protocols with exponentially smaller complexity. For set-disjointness, which also requires linear randomized communication complexity in the two-party model, we give a deterministic protocol for the asymmetric version in his model with a quadratic improvement in complexity. These results demonstrate that Patrascu's conjecture, as stated, is false. In addition, we show that the randomized and deterministic communication complexities of problems in his model differ by no more than a logarithmic multiplicative factor.We also prove lower bounds in some restricted versions of this model for natural functions such as set-disjointness and inner product. All of our upper bounds conform to these restrictions.

We consider the setting of a multiprocessor where the speeds of the m processors can be individually scaled. Jobs arrive over time and have varying degrees of parallelizability. A nonclairvoyant scheduler must assign the jobs to processors, and scale the speeds of the processors. We consider the objective of energy plus flow time. For jobs that may have side effects or that are not checkpointable, we show an Ω( m (α--1)/α2) bound on the competitive ratio of any deterministic algorithm. Here m is the number of processors and α is the exponent of the power function. For checkpointable jobs without side effects, we give an O (log m )-competitive algorithm. Thus for jobs that may have side effects or that are not checkpointable, the achievable competitive ratio grows quickly with the number of processors, but for checkpointable jobs without side effects, the achievable competitive ratio grows slowly with the number of processors. We then show a lower bound of Ω(log1/α m ) on the competitive ratio of any algorithm for checkpointable jobs without side effects. Finally we slightly improve the upper bound on the competitive ratio for the single processor case, which is equivalent to the case that all jobs are fully parallelizable, by giving an improved analysis of a previously proposed algorithm.

We give three results related to online nonclairvoyant speed scaling to minimize total flow time plus energy. We give a nonclairvoyant algorithm LAPS, and show that for every power function of the form P ( s )= s α , LAPS is O (1)-competitive; more precisely, the competitive ratio is 8 for α =2, 13 for α =3, and $\frac{2\alpha^{2}}{\ln\alpha}$ for α >3. We then show that there is no constant c , and no deterministic nonclairvoyant algorithm A , such that A is c -competitive for every power function of the form P ( s )= s α . So necessarily the achievable competitive ratio increases as the steepness of the power function increases. Finally we show that there is a fixed, very steep, power function for which no nonclairvoyant algorithm can be O (1)-competitive.

We address the scheduling model of arbitrary speed-up curves and the broadcast scheduling model. The former occurs when jobs are scheduled in a multi-core system or on a cloud of machines. Here jobs can be sped up when given more processors or machines. However, the parallelizability of the jobs may vary and the algorithm is required to be oblivious of the parallelizability of a job. The latter model is natural in wireless and LAN networks where requests (or jobs) can be simultaneously satisfied together. Both settings are similar in that two schedules can do different amounts of work to satisfy all the jobs. We focus on optimizing the lk- norms of flow time. Recently, Gupta et al. [24] gave a (k + ε)-speed O(1)-competitive algorithm for the lk norms of flow time in both scheduling settings for fixed k. Inspired by this work, we give the first analysis of a scalable algorithm, i.e. (1 + ε)-speed O(1)-competitive, for all lk-norms of flow time in both settings for fixed k and 0 < ε ≤ 1. Both problems have a strong lower bound without resource augmentation, so this is the best result that can be shown in the worst case setting up to a constant factor in the competitive ratio.

The goal is to prove a surprising lower bound for resource augmented nonclairvoyant algorithms for scheduling jobs with sublinear nondecreasing speed-up curves on multiple processors with the objective of average response time. Edmonds and Pruhs in SODA09 prove that for every $\e > 0$, there is an algorithm $\alg_{\e}$ that is $(1\!+\!\epsilon)$-speed $O({1 \over \e2})$-competitive. A problem, however, is that this algorithm $\alg_{\e}$ depends on $\e$. The goal is to prove that every fixed deterministic nonclairvoyant algorithm has a suboptimal speed threshold, namely for every (graceful) algorithm $\alg$, there is a threshold $1\!+\!\beta_{\alg}$ that is $\beta_{\alg} > 0$ away from being optimal such that the algorithm is $\Omega({1 \over \e \beta_{\alg}})$ competitive with speed $(1 \!+\! \beta_{\alg}) \!+\! \e$ and is $\omega(1)$ competitive with speed $1 \!+\! \beta_{\alg}$. I have worked very hard on it and have felt that I was close. The proof technique is to use Brouwer's fixed point theorem to break the cycle of needing to know which input will be given before one can know what the algorithm will do and needing to know what the algorithm will do before one can know which input to give. Every thing I have can be found at

We consider the problem of computing bounds on the variance and expectation of the longest path length in a DAG from knowledge of variance and expectation of edge lengths. We focus primarily on the case where all edge lengths are non-negative and the DAG has a single source and sink node. We present analytic bounds for various simple DAG structures, and present a new algorithm to compute bounds for more general DAG structures. Our algorithm is motivated by an analogy with balance of forces in a network of strange springs.

The well-known Transport Control Protocol (TCP) is a crucial component of the TCP/IP architecture on which the Internet is built, and is a de facto standard for reliable communication on the Internet. At the heart of the TCP protocol is its congestion control algorithm. While most practitioners believe that the TCP congestion control algorithm performs very well, a complete analysis of the congestion control algorithm is yet to be done. A lot of effort has, therefore, gone into the evaluation of different performance metrics like throughput and average latency under TCP. In this paper, we approach the problem from a different perspective and use the competitive analysis framework to provide some answers to the question “how good is the TCP/IP congestion control algorithm?” We describe how the TCP congestion control algorithm can be viewed as an online, distributed scheduling algorithm. We observe that existing lower bounds for non-clairvoyant scheduling algorithms imply that no online, distributed, non-clairvoyant algorithm can be competitive with an optimal offline algorithm if both algorithms were given the same resources. Therefore, in order to evaluate TCP using competitive analysis, we must limit the power of the adversary, or equivalently, allow TCP to have extra resources compared to an optimal, offline algorithm for the same problem. In this paper, we show that TCP is competitive to an optimal, offline algorithm provided the former is given more resources. Specifically, we prove first that for networks with a single bottleneck (or point of congestion), TCP is \({\mathcal{O}}(1)\)-competitive to an optimal centralized (global) algorithm in minimizing the user-perceived latency or flow time of the sessions, provided we allow TCP \({\mathcal{O}}(1)\) times as much bandwidth and \({\mathcal{O}}(1)\) extra time per session. Second, we show that TCP is fair by proving that the bandwidths allocated to sessions quickly converge to fair sharing of network bandwidth.

We consider the problem of computing a minimum-distortion bijection between two point-sets in R 2 . We prove the first non-trivial inapproximability result for this problem, for the case when the distortion is constant. More precisely, we show that there exist constants 0 < α < β, such that it is NP-hard to distinguish between spaces for which the distortion is either at most α, or at least β, under the Euclidean norm. This addresses a question of Kenyon, Rabani and Sinclair [KRS04], and extends a result due to Papadimitriou and Safra [PS05], who gave inapproximability for point-sets in R 3 . We also apply similar ideas to the problem of computing a minimum-distortion embedding of a finite metric space into R 2 . We obtain an analogous inapproximability result under the l ∞ norm for this problem. Inapproximability for the case of constant distortion was previously known only for dimension at least 3 [MS08].