Sampling a random walk is a fundamental primitive in many graph applications. In the streaming model, it is known that sampling an L-step random walk on an n-vertex directed graph requires Omega(nL) space, implying that no sublinear-space streaming algorithm exists for general graphs. We show that sublinear algorithms are possible for the case of dense graphs, where every vertex has out-degree at least Omega(n). In particular, we give a one-pass turnstile streaming algorithm that uses only (O) over tilde (L) memory for such graphs. More broadly, for graphs with minimum out-degree at least d, our streaming algorithm samples a random walk using (O) over tilde ( n/d center dot L) memory. We show that our algorithm is optimal in a strong "beyond worst-case" sense. To formalize this, we introduce the notion of universal optimality for graph streaming algorithms. Informally, a streaming algorithm is universally optimal if it performs (almost) as well as possible on every graph, assuming a worst-case choice of the streaming order. This notion of universal optimality is a key conceptual contribution of our work. 2012 ACM Subject Classification Theory of computation -> Streaming, sublinear and near linear time algorithms

We introduce a variant of Error Correcting Codes with no predetermined length. An Unbounded ECC with rate R and distance ε is an encoding of a possibly infinite message into a possibly infinite codeword, such that for every large enough k we may recover the first Rk symbols of the message from the first k symbols of the codeword – even when up to 1/2ε k of these codeword symbols are adversarially corrupted. We study unbounded codes over a binary alphabet in the regime of small distance ε, and obtain nearly-tight upper and lower bounds in several natural settings. We show that the optimal rate of such a code is between R<1-Ω(√(ε)) and R>1-O(√(εloglog(1/ε))). Surprisingly, our construction is non-linear, and we show that the optimal rate of a linear unbounded code is the asymptotically worse R=1-Θ(√(εlog(1/ε))). In the setting of random noise, the optimal rate of unbounded codes improves and matches the rate of standard codes at R=1-Θ(εlog(1/ε)).

The Massart noise condition is a central model in Probably Approximately Correct (PAC) learning theory. Its importance lies in it being an interpolation condition between realizable and the agnostic settings, under which one can attain faster rates than latter, and, under strict conditions, recover the rates of the former. Despite its importance, the Massart condition has not yet been fully explored in emerging extensions of statistical learning theory beyond the classical PAC framework. In this work, we present two such extensions. First, we revisit the transductive empirical risk minimization (TERM) algorithm of (Hanneke & Moran, 2026) and derive sharper excess error bounds under Massart noise using offset Rademacher techniques and local metric entropy introduced by (Zhivotovskiy & Hanneke, 2018). We then leverage this analysis to obtain new optimal sample complexity bounds for PAC learning with partial concept classes and complete the characterization of universal learning rates under Massart noise.

In Bernoulli discriminative models, log-likelihood is a natural — and, in a well-defined sense, universal — choice of risk score. In this general setting, we propose a non-parametric variant of binary regression, where the model is regularized to be a Lipschitz function taking a metric space to [0, 1]. Our choice of logarithmic loss corresponds to the log-likelihood risk score. This setting presents novel computational and statistical challenges. On the computational front, we derive an efficient optimization algorithm based on interior point methods (IPM); an attractive feature is that it is parameter-free (that is, does not require tuning an update step size). On the statistical front, the unbounded loss function presents a problem for classic generalization bounds, based on covering-number and Rademacher techniques. Additionally, an impossibility result we prove shows that the unboundedness presents an inherent obstruction to learnability. We get around this challenge via an adaptive truncation approach, and also derive a lower bound indicating that the truncation is, in some sense, necessary. To our knowledge, our approach provides the first rigorous computational and theoretical results in this area. Finally, we present encouraging empirical results.

The notion of closure of a set of linear forms, first introduced by Efremenko, Garlik, and Itsykson [14], has proven instrumental in proving lower bounds on the sizes of regular and bounded-depth Res(circle plus) refutations [14, 3]. In this work, we present amortized closure, an enhancement that retains the properties of original closure [14] but offers tighter control on its growth. Specifically, adding a new linear form increases the amortized closure by at most one. We explore two applications that highlight the power of this new concept. Utilizing our newly defined amortized closure, we extend and provide a succinct and elegant proof of the recent lifting theorem by Chattopadhyay and Dvorak [10]. Namely we show that for an unsatisfiable CNF formula phi and a 1-stifling gadget g : {0, 1} (l)->{0, 1}, if the lifted formula phi o g has a tree-like Res(circle plus) refutation of size 2(d) and width w, then phi has a resolution refutation of depth d and width w. The original theorem by Chattopadhyay and Dvorak [10] applies only to the more restrictive class of strongly stifling gadgets. As a more significant application of amortized closure, we show improved lower bounds for bounded-depth Res(circle plus), extending the depth beyond that of Alekseev and Itsykson [3]. Our result establishes an exponential lower bound for depth-Omega( n log n) Res(circle plus) refutations of lifted Tseitin formulas, a notable improvement over the existing depth-Omega( n log log n) Res(circle plus) lower bound.

In a celebrated result from the 60's, Berlekamp showed that feedback can be used to increase the maximum fraction of adversarial noise that can be tolerated by binary error correcting codes from (1)/(4) to (1)/(3). However, his result relies on the assumption that feedback is "continuous", i.e., after every utilization of the channel, the sender gets the symbol received by the receiver. While this assumption is natural in some settings, in other settings it may be unreasonable or too costly to maintain. In this work, we initiate the study of round-restricted feedback channels, where the number r of feedback rounds is possibly much smaller than the number of utilizations of the channel. Error correcting codes for such channels are protocols where the sender can ask for feedback at most r times, and, upon a feedback request, it obtains all the symbols received since its last feedback request. We design such error correcting protocols for both the adversarial binary erasure channel and for the adversarial binary corruption (bit flip) channel. For the erasure channel, we give an exact characterization of the round-vs-resilience tradeoff by designing a (constant rate) protocol with r feedback rounds, for every r, and proving that its noise resilience is optimal. Designing such error correcting protocols for the corruption channel is substantially more involved. We show that obtaining the optimal resilience, even with one feedback round ( r = 1), requires settling (proving or disproving) a new, seemingly unrelated, "clean" combinatorial conjecture, about the maximum cut in weighted graphs versus the "imbalance" of an average cut. Specifically, we prove an upper bound on the optimal resilience (impossibility result), and show that the existence of a matching lower bound (a protocol) is equivalent to the correctness of our conjecture.

A knockout tournament is one of the most simple and popular forms of competition. Here, we are given a binary tournament tree where all leaves are labeled with seed position names. The players participating in the tournament are assigned to the seed positions. In each round, the two players assigned to leaves of the tournament tree with a common parent compete, and the winner is promoted to the parent. The last remaining player is the winner of the tournament. In this work, we study the problem of making knock-out tournaments robust against manipulation, where the form of manipulation we consider is changing the outcome of a game. We assume that our input is only the number of players that compete in the tournament, and the number of manipulations against which the tournament should be robust. Furthermore, we assume that there is a strongest player, that is, a player that beats any of the other players. However, the identity of this player is not part of the problem input. To ensure robustness against manipulation, we uncover an unexpected connection between the problem at hand and communication protocols that utilize a feedback channel, offering resilience against adversarial noise. We explore the trade-off between the size of the robust tournament tree and the degree of protection against manipulation. Specifically, we demonstrate that it is possible to tolerate up to a 1/3 fraction of manipulations along each leaf-to-root path, at the cost of only a polynomial blow-up in the tournament size.

Can the n-party broadcast channel, where any symbol sent by one party is received by all, be made resilient to noise with low overhead? Namely, is it possible to construct interactive error-correcting codes that convert any protocol designed for the noiseless broadcast channel into one that works over the noisy broadcast channel and is not much longer than the original protocol? [12, STOC 2018] showed that such interactive codes with constant multiplicative overhead are possible under the assumption that the noiseless protocol being simulated is non-adaptive, meaning that it is restricted to have a pre-determined order of turns. Their noise resilient simulating protocols, however, require adaptivity, where each party can decide whether or not to broadcast given all the information available to them, including their input and received transcript. The question of whether such a simulation is possible for general, potentially adaptive, noiseless protocols was left open. We resolve this question negatively, proving that any interactive code that converts adaptive noiseless broadcast protocols into adaptive broadcast protocols resilient to stochastic errors must incur a multiplicative overhead of Omega(log n/log log n), which is nearly tight.

The proof system resolution over parities (Res(circle plus)) operates with disjunctions of linear equations (linear clauses) over F-2; it extends the resolution proof system by incorporating linear algebra over F-2. Over the years, several exponential lower bounds on the size of tree-like Res(circle plus) refutations have been established. However, proving a superpolynomial lower bound on the size of dag-like Res(circle plus) refutations remains a highly challenging open question. We prove an exponential lower bound for regular Res(circle plus). Regular Res(circle plus) is a subsystem of dag-like Res(circle plus) that naturally extends regular resolution. This is the first known superpolynomial lower bound for a fragment of dag-like Res(circle plus) which is exponentially stronger than tree-like Res(circle plus). In the regular regime, resolving linear clauses C-1 and C-2 on a linear form f is permitted only if, for both i is an element of{1, 2}, the linear form f does not lie within the linear span of all linear forms that were used in resolution rules during the derivation of C-i. Namely, we show that the size of any regular Res(circle plus) refutation of the binary pigeonhole principle BPHPnn+1 is at least 2(Omega(3 root n/ log n)). A corollary of our result is an exponential lower bound on the size of a strongly read-once linear branching program solving a search problem. This resolves an open question raised by Gryaznov, Pudlak, and Talebanfard [Proceedings of the 37th Computational Complexity Conference, LIPIcs Leibniz Int. Proc. Inform. 234, S. Lovett, ed., Schloss Dagstuhl-Leibniz-Zentrum fur Informatik, 2022, pp. 1-16]. As a byproduct of our technique, we prove that the size of any tree-like Res(circle plus) refutation of the weak binary pigeonhole principle BPHPnm is at least 2(Omega(n)) using Prover-Delayer games. We also give a direct proof of a width lower bound: we show that any dag-like Res(circle plus) refutation of BPHPnm contains a linear clause C with Omega(n) linearly independent equations.

We initiate the study of error correcting codes over the multi-party adversarial broadcast channel. Specifically, we consider the classic information dissemination problem where n parties, each holding an input bit, wish to know each other's input. For this, they communicate in rounds, where, in each round, one designated party sends a bit to all other parties over a channel governed by an adversary that may corrupt a constant fraction of the received communication. We mention that the dissemination problem was studied in the stochastic noise model since the 80's. While stochastic noise in multi-party channels has received quite a bit of attention, the case of adversarial noise has largely been avoided, as such channels cannot handle more than a 1 n -fraction of errors. Indeed, this many errors allow an adversary to completely corrupt the incoming or outgoing communication for one of the parties and fail the protocol. Curiously, we show that by eliminating these "trivial" attacks, one can get a simple protocol resilient to a constant fraction of errors. Thus, a model that rules out such attacks is both necessary and sufficient to get a resilient protocol. The main shortcoming of our dissemination protocol is its length: it requires Theta(n(2)) communication rounds whereas n rounds suffice in the absence of noise. Our main result is a matching lower bound of Omega(n(2)) on the length of any dissemination protocol in our model. Our proof first "gets rid" of the channel noise by converting it to a form of "input noise", showing that a noisy dissemination protocol implies a (noiseless) protocol for a version of the direct sum gap-majority problem. We conclude the proof with a tight lower bound for the latter problem, which may be of independent interest.

Single-hop radio networks (SHRN) are a well studied abstraction of communication over a wireless channel. In this model, in every round, each of the n participating parties may decide to broadcast a message to all the others, potentially causing collisions. We consider the SHRN model in the presence of stochastic message drops (i.e., erasures), where in every round, the message received by each party is erased (replaced by perpendicular to) with some small constant probability, independently. Our main result is a constant rate coding scheme, allowing one to run protocols designed to work over the (noiseless) SHRN model over the SHRN model with erasures. Our scheme converts any protocol Pi of length at most exponential in n over the SHRN model to a protocol Pi ' that is resilient to constant fraction of erasures and has length linear in the length of Pi. We mention that for the special case where the protocol Pi is non-adaptive, i.e., the order of communication is fixed in advance, such a scheme was known. Nevertheless, adaptivity is widely used and is known to hugely boost the power of wireless channels, which makes handling the general case of adaptive protocols. both important and more challenging. Indeed, to the best of our knowledge, our result is the first constant rate scheme that converts adaptive protocols to noise resilient ones in any multi-party model.

Kol and Raz [STOC 2013] showed how to simulate any alternating two-party communication protocol designed to work over the noiseless channel, by a protocol that works over a stochastic channel that corrupts each sent symbol with probability epsilon > 0 independently, with only a 1 + O(root H(epsilon)) blowup to the communication. In particular, this implies that the maximum rate of such interactive codes approaches 1 as epsilon goes to 0, as is also the case for the maximum rate of classical error correcting codes. Over the past decade, followup works have strengthened and generalized this result to other noisy channels, stressing on how fast the rate approaches 1 as epsilon goes to 0, but retaining the assumption that the noiseless protocol is alternating. In this paper we consider the general case, where the noiseless protocols can have arbitrary orders of speaking. In contrast to Kol-Raz and to the followup results in this model, we show that the maximum rate of interactive codes that encode general protocols is upper bounded by a universal constant strictly smaller than 1. To put it differently, we show that there is an inherent blowup in communication when protocols with arbitrary orders of speaking are faced with any constant fraction of errors epsilon > 0. We mention that our result assumes a large alphabet set and resolves the (non-binary variant) of a conjecture by Haeupler [FOCS 2014].

In this work, we design an interactive coding scheme that converts any two party interactive protocol Pi into another interactive protocol Pi', such that even if errors are introduced during the execution of Pi', the parties are able to determine what the outcome of running Pi would be in an error-free setting. Importantly, our scheme preserves the space complexity of the protocol, in addition to the communication and computational complexities. Specifically, if the protocol Pi has communication complexity T, computational complexity t, and space complexity s, the resulting protocol Pi' is resilient to a constant epsilon > 0 fraction of adversarial errors, and has communication complexity approaching T as epsilon approaches 0, computational complexity poly(t), and space complexity O(s log T). Prior to this work, all known interactive coding schemes required the parties to use at least Omega(T) space, as the parties were required to remember the transcript of the conversation thus far, or considered weaker error models.

Hypothesis Selection is a fundamental distribution learning problem where given a comparator-class $\mathcal{Q}=\{q_{1}, \ldots, q_{n}\}$ of distributions, and a sampling access to an unknown target distribution $p$ , the goal is to output a distribution $q$ such that $\mathsf{TV}(p, q)$ is close to opt, where $\mathsf{opt}=\min\nolimits_{i}\{\mathsf{TV}(p, q_{i})\}$ and TV (.,.) denotes the total-variation distance. Despite the fact that this problem has been studied since the 19th century, its complexity in terms of basic resources, such as number of samples and approximation guarantees, remains unsettled (this is discussed, e.g., in the charming book by Devroye and Lugosi '00). This is in stark contrast with other (younger) learning settings, such as PAC learning, for which these complexities are well understood. We derive an optimal 2-approximation learning strategy for the Hypothesis Selection problem, outputting $q$ such that, $\mathsf{TV}(p, q)\leq 2\cdot\text{opt}+\varepsilon$ , with a (nearly) optimal sample complexity of $\tilde{O}(\log n/\varepsilon^{2})$ . This is the first algorithm that simultaneously achieves the best approximation factor and sample complexity: previously, Bousquet, Kane, and Moran (COLT ‘19) gave a learner achieving the optimal 2-approximation, but with an exponentially worse sample complexity of $\tilde{O}(\sqrt{n}/\varepsilon^{2.5})$ , and Yatracos (Annals of Statistics '85) gave a learner with optimal sample complexity of $O(\log n/\varepsilon^{2})$ but with a sub-optimal approximation factor of 3. We mention that many works in the Density Estimation (a.k.a., Distribution Learning) literature use Hypothesis Selection as a black box subroutine. Our result therefore implies an improvement on the approximation factors obtained by these works, while keeping their sample complexity intact. For example, our result improves the approximation factor of the algorithm of Ashtiani, Ben-David, Harvey, Liaw, and Mehrabian (JACM '20) for agnostic learning of mixtures of gaussians from 9 to 6, while maintaining its nearly-tight sample complexity.

Let II be a protocol over the n-party broadcast channel, where in each round, a pre-specified party broadcasts a symbol to all other parties. We wish to design a scheme that takes such a protocol II as input and outputs a noise resilient protocol II’ that simulates II over the noisy broadcast channel, where each received symbol is flipped with a fixed constant probability, independently. What is the minimum overhead in the number of rounds that is incurred by any such simulation scheme? A classical result by Gallager from the 80's shows that non-interactive T-round protocols, where the bit communicated in every round is independent of the communication history, can be converted to noise resilient ones with only an $\mathrm{O}(\log\log T$) multiplicative overhead in the number of rounds. Can the same be proved for any protocol? Or, are there protocols whose simulation requires an $\Omega(\log T)$ overhead (which always suffices)? We answer both the above questions in the negative: We give a simulation scheme with an $\tilde{O}(\sqrt{\log T})$ overhead for every protocol and channel alphabet. We also prove an (almost) matching lower bound of $\Omega(\sqrt{\log T})$ on the overhead required to simulate the pointer chasing protocol with T = n and polynomial alphabet.

Given a Boolean circuit C , we wish to convert it to a circuit C ′ that computes the same function as C even if some of its gates suffer from adversarial short circuit errors, i.e., their output is replaced by the value of one of their inputs. Can we design such a resilient circuit C ′ whose size is roughly comparable to that of C ? Prior work gave a positive answer for the special case where C is a formula. We study the general case and show that any Boolean circuit C of size s can be converted to a new circuit C ′ of quasi-polynomial size s O (log s ) that computes the same function as C even if a 1/51 fraction of the gates on any root-to-leaf path in C ′ are short circuited. Moreover, if the original circuit C is a formula, the resilient circuit C ′ is of near-linear size s 1+є . The construction of our resilient circuits utilizes the connection between circuits and DAG-like communication protocols, originally introduced in the context of proof complexity.

We consider fault-tolerant boolean formulas in which the output of a faulty gate is short-circuited to one of the gate's inputs. A recent result by Kalai et al. [FOCS 2012] converts any boolean formula into a resilient formula of polynomial size that works correctly if less than a fraction 1/6 of the gates (on every input-to-output path) are faulty. We improve the result of Kalai et al., and show how to efficiently fortify any boolean formula against a fraction 1/5 of short-circuit gates per path, with only a polynomial blowup in size. We additionally show that it is impossible to obtain formulas with higher resilience and sub-exponential growth in size. Towards our results, we consider interactive coding schemes when noiseless feedback is present; these produce resilient boolean formulas via a Karchmer-Wigderson relation. We develop a coding scheme that resists up to a fraction 1/5 of corrupted transmissions in each direction of the interactive channel. We further show that such a level of noise is maximal for coding schemes with sub-exponential blowup in communication. Our coding scheme takes a surprising inspiration from Blockchain technology.

In the reliable transmission problem, a sender, Alice, wishes to transmit a bit-string x to a remote receiver, Bob, over a binary channel with adversarial noise. The solution to this problem is to encode x using an error correcting code. As it is long known that the distance of binary codes is at most 1/2, reliable transmission is possible only if the channel corrupts (flips) at most a 1/4-fraction of the communicated bits.We revisit the reliable transmission problem in the two-way setting, where both Alice and Bob can send bits to each other. Our main result is the construction of two-way error correcting codes that are resilient to a constant fraction of corruptions strictly larger than 1/4. Moreover, our code has constant rate and requires Bob to only send one short message. We mention that our result resolves an open problem by Haeupler, Kamath, and Velingker [APPROX-RANDOM, 2015] and by Gupta, Kalai, and Zhang [STOC, 2022].Curiously, our new two-way code requires a fresh perspective on classical error correcting codes: While classical codes have only one distance guarantee for all pairs of codewords (i.e., the minimum distance), we construct codes where the distance between a pair of codewords depends on the “compatibility” of the messages they encode. We also prove that such codes are necessary for our result.