How is knowledge stored in an LLM’s weights? We study this via layer pruning: if removing a certain layer does not affect model performance in common question-answering benchmarks, then the weights in that layer are not necessary for storing the knowledge needed to answer those questions. To find these unnecessary parameters, we identify the optimal block of layers to prune by considering similarity across layers; then, to “heal” the damage, we perform a small amount of finetuning. Surprisingly, with this method we find minimal degradation of performance until after a large fraction (up to half) of the layers are removed for some common open-weight models. From a scientific perspective, the robustness of these LLMs to the deletion of layers implies either that current pretraining methods are not properly leveraging the parameters in the deeper layers of the network or that the shallow layers play a critical role in storing knowledge. For our study, we use parameter-efficient finetuning (PEFT) methods, specifically quantization and Low Rank Adapters (QLoRA), such that each of our experiments can be performed on a single 40GB A100 GPU.

Fully-connected deep neural networks with weights initialized from independent Gaussian distributions can be tuned to criticality, which prevents the exponential growth or decay of signals propagating through the network. However, such networks still exhibit fluctuations that grow linearly with the depth of the network, which may impair the training of networks with width comparable to depth. We show analytically that rectangular networks with tanh activations and weights initialized from the ensemble of orthogonal matrices have corresponding preactivation fluctuations which are independent of depth, to leading order in inverse width. Moreover, we demonstrate numerically that, at initialization, all correlators involving the neural tangent kernel (NTK) and its descendants at leading order in inverse width -- which govern the evolution of observables during training -- saturate at a depth of $\sim 20$, rather than growing without bound as in the case of Gaussian initializations. We speculate that this structure preserves finite-width feature learning while reducing overall noise, thus improving both generalization and training speed in deep networks with depth comparable to width. We provide some experimental justification by relating empirical measurements of the NTK to the superior performance of deep nonlinear orthogonal networks trained under full-batch gradient descent on the MNIST and CIFAR-10 classification tasks.

The proliferation of generative models, combined with pretraining on web-scale data, raises a timely question: what happens when these models are trained on their own generated outputs? Recent investigations into model-data feedback loops discovered that such loops can lead to model collapse, a phenomenon where performance progressively degrades with each model-fitting iteration until the latest model becomes useless. However, several recent papers studying model collapse assumed that new data replace old data over time rather than assuming data accumulate over time. In this paper, we compare these two settings and show that accumulating data prevents model collapse. We begin by studying an analytically tractable setup in which a sequence of linear models are fit to the previous models' predictions. Previous work showed if data are replaced, the test error increases linearly with the number of model-fitting iterations; we extend this result by proving that if data instead accumulate, the test error has a finite upper bound independent of the number of iterations. We next empirically test whether accumulating data similarly prevents model collapse by pretraining sequences of language models on text corpora. We confirm that replacing data does indeed cause model collapse, then demonstrate that accumulating data prevents model collapse; these results hold across a range of model sizes, architectures and hyperparameters. We further show that similar results hold for other deep generative models on real data: diffusion models for molecule generation and variational autoencoders for image generation. Our work provides consistent theoretical and empirical evidence that data accumulation mitigates model collapse.

Large language models with a huge number of parameters, when trained on near internet-sized number of tokens, have been empirically shown to obey neural scaling laws: specifically, their performance behaves predictably as a power law in either parameters or dataset size until bottlenecked by the other resource. To understand this better, we first identify the necessary properties allowing such scaling laws to arise and then propose a statistical model -- a joint generative data model and random feature model -- that captures this neural scaling phenomenology. By solving this model in the dual limit of large training set size and large number of parameters, we gain insight into (i) the statistical structure of datasets and tasks that lead to scaling laws, (ii) the way nonlinear feature maps, such as those provided by neural networks, enable scaling laws when trained on these datasets, (iii) the optimality of the equiparameterization scaling of training sets and parameters, and (iv) whether such scaling laws can break down and how they behave when they do. Key findings are the manner in which the power laws that occur in the statistics of natural datasets are extended by nonlinear random feature maps and then translated into power-law scalings of the test loss and how the finite extent of the data's spectral power law causes the model's performance to plateau.

Autoencoders have been proposed as a powerful tool for model-independent anomaly detection in high-energy physics. The operating principle is that events which do not belong to the space of training data will be reconstructed poorly, thus flagging them as anomalies. We point out that in a variety of examples of interest, the connection between large reconstruction error and anomalies is not so clear. In particular, for data sets with nontrivial topology, there will always be points that erroneously seem anomalous due to global issues. Conversely, neural networks typically have an inductive bias or prior to locally interpolate such that undersampled or rare events may be reconstructed with small error, despite actually being the desired anomalies. Taken together, these facts are in tension with the simple picture of the autoencoder as an anomaly detector. Using a series of illustrative low-dimensional examples, we show explicitly how the intrinsic and extrinsic topology of the dataset affects the behavior of an autoencoder and how this topology is manifested in the latent space representation during training. We ground this analysis in the discussion of a mock "bump hunt" in which the autoencoder fails to identify an anomalous "signal" for reasons tied to the intrinsic topology of $n$-particle phase space.

We discuss why AI is hard and why physics is simple. We discuss how physical intuition and the approach of theoretical physics can be brought to bear on the field of artificial intelligence and specifically machine learning. We suggest that the underlying project of machine learning and the underlying project of physics are strongly coupled through the principle of sparsity, and we call upon theoretical physicists to work on AI as physicists. As a first step in that direction, we discuss an upcoming book on the principles of deep learning theory that attempts to realize this approach.

This book develops an effective theory approach to understanding deep neural networks of practical relevance. Beginning from a first-principles component-level picture of networks, we explain how to determine an accurate description of the output of trained networks by solving layer-to-layer iteration equations and nonlinear learning dynamics. A main result is that the predictions of networks are described by nearly-Gaussian distributions, with the depth-to-width aspect ratio of the network controlling the deviations from the infinite-width Gaussian description. We explain how these effectively-deep networks learn nontrivial representations from training and more broadly analyze the mechanism of representation learning for nonlinear models. From a nearly-kernel-methods perspective, we find that the dependence of such models' predictions on the underlying learning algorithm can be expressed in a simple and universal way. To obtain these results, we develop the notion of representation group flow (RG flow) to characterize the propagation of signals through the network. By tuning networks to criticality, we give a practical solution to the exploding and vanishing gradient problem. We further explain how RG flow leads to near-universal behavior and lets us categorize networks built from different activation functions into universality classes. Altogether, we show that the depth-to-width ratio governs the effective model complexity of the ensemble of trained networks. By using information-theoretic techniques, we estimate the optimal aspect ratio at which we expect the network to be practically most useful and show how residual connections can be used to push this scale to arbitrary depths. With these tools, we can learn in detail about the inductive bias of architectures, hyperparameters, and optimizers.

We derive a simple and model-independent formula for the change in the generalization gap due to a gradient descent update. We then compare the change in the test error for stochastic gradient descent to the change in test error from an equivalent number of gradient descent updates and show explicitly that stochastic gradient descent acts to regularize generalization error by decorrelating nearby updates. These calculations depends on the details of the model only through the mean and covariance of the gradient distribution, which may be readily measured for particular models of interest. We discuss further improvements to these calculations and comment on possible implications for stochastic optimization.

Design of reliable systems must guarantee stability against input perturbations. In machine learning, such guarantee entails preventing overfitting and ensuring robustness of models against corruption of input data. In order to maximize stability, we analyze and develop a computationally efficient implementation of Jacobian regularization that increases classification margins of neural networks. The stabilizing effect of the Jacobian regularizer leads to significant improvements in robustness, as measured against both random and adversarial input perturbations, without severely degrading generalization properties on clean data.

We show that in a variety of large-scale deep learning scenarios the gradient dynamically converges to a very small subspace after a short period of training. The subspace is spanned by a few top eigenvectors of the Hessian (equal to the number of classes in the dataset), and is mostly preserved over long periods of training. A simple argument then suggests that gradient descent may happen mostly in this subspace. We give an example of this effect in a solvable model of classification, and we comment on possible implications for optimization and learning.

We discuss the probability distribution for the “size” of a time-evolving operator in the SYK model. Scrambling is related to the fact that as time passes, the distribution shifts towards larger operators. Initially, the rate is exponential and determined by the infinite-temperature chaos exponent. We evaluate the size distribution numerically for N = 30, and show how to compute it in the large-N theory using the dressed fermion propagator. We then evaluate the distribution explicitly at leading nontrivial order in the large-q expansion.

We study the relationship between quantum chaos and pseudorandomness by developing probes of unitary design. A natural probe of randomness is the "frame potential," which is minimized by unitary $k$-designs and measures the $2$-norm distance between the Haar random unitary ensemble and another ensemble. A natural probe of quantum chaos is out-of-time-order (OTO) four-point correlation functions. We show that the norm squared of a generalization of out-of-time-order $2k$-point correlators is proportional to the $k$th frame potential, providing a quantitative connection between chaos and pseudorandomness. Additionally, we prove that these $2k$-point correlators for Pauli operators completely determine the $k$-fold channel of an ensemble of unitary operators. Finally, we use a counting argument to obtain a lower bound on the quantum circuit complexity in terms of the frame potential. This provides a direct link between chaos, complexity, and randomness.

The first generation of quantum information applications are *trusted* solutions: when a quantum state needs to be created or a quantum operation needs to be applied, the user simply believes that his/her quantum hardware executes the instructions according to the specifications. The past few years have witnessed the rapid development of a new generation of quantum information applications. They are *trustworthy* in the sense that a quantum device has to prove to the user that it is operating as expected. Trustworthy quantum information enables verification of potentially faulty quantum devices and provides resilience against adversarially prepared quantum devices. These practical motivations often lead to fundamental questions about quantum information. In this talk, I will present some of the highlights in this emerging area and invite the audience to research the many fundamental and challenging open problems.

As experiments are increasingly able to probe the quantum dynamics of systems with many degrees of freedom, it is interesting to probe fundamental bounds on the dynamics of quantum information. We elaborate on the relationship between one such bound-the Lieb-Robinson bound-and the butterfly effect in strongly coupled quantum systems. The butterfly effect implies the ballistic growth of local operators in time, which can be quantified with the "butterfly" velocity v_{B}. Similarly, the Lieb-Robinson velocity places a state-independent ballistic upper bound on the size of time evolved operators in nonrelativistic lattice models. Here, we argue that v_{B} is a state-dependent effective Lieb-Robinson velocity. We study the butterfly velocity in a wide variety of quantum field theories using holography and compare with free-particle computations to understand the role of strong coupling. We find that v_{B} remains constant or decreases with decreasing temperature. We also comment on experimental prospects and on the relationship between the butterfly velocity and signaling.

We study chaos and scrambling in unitary channels by considering their entanglement properties as states. Using out-of-time-order correlation functions to diagnose chaos, we characterize the ability of a channel to process quantum information. We show that the generic decay of such correlators implies that any input subsystem must have near vanishing mutual information with almost all partitions of the output. Additionally, we propose the negativity of the tripartite information of the channel as a general diagnostic of scrambling. This measures the delocalization of information and is closely related to the decay of out-of-time-order correlators. We back up our results with numerics in two non-integrable models and analytic results in a perfect tensor network model of chaotic time evolution. These results show that the butterfly effect in quantum systems implies the information-theoretic definition of scrambling.

We conjecture that the quantum complexity of a holographic state is dual to the action of a certain spacetime region that we call a Wheeler-DeWitt patch. We illustrate and test the conjecture in the context of neutral, charged, and rotating black holes in anti-de Sitter spacetime, as well as black holes perturbed with static shells and with shock waves. This conjecture evolved from a previous conjecture that complexity is dual to spatial volume, but appears to be a major improvement over the original. In light of our results, we discuss the hypothesis that black holes are the fastest computers in nature.

We study chaotic dynamics in two-dimensional conformal field theory through out-of-time order thermal correlators of the form ⟨ W(t)VW(t)V⟩. We reproduce bulk calculations similar to those of [1], by studying the large c Virasoro identity block. The contribution of this block to the above correlation function begins to decrease exponentially after a delay of ∼ t_* - β/2πlogβ^2E_w E_v, where t_* is the scrambling time β/2πlog c, and E_w,E_v are the energy scales of the W,V operators.

Our earlier paper "Complexity Equals Action" conjectured that the quantum computational complexity of a holographic state is given by the classical action of a region in the bulk (the "Wheeler-DeWitt" patch). We provide calculations for the results quoted in that paper, explain how it fits into a broader (tensor) network of ideas, and elaborate on the hypothesis that black holes are the fastest computers in nature.

We study products of precursors of spatially local operators, \( {W_x}_{{}_n}(tn)\cdot \cdot \cdot {W}_{x_1}\left({t}_1\right) \), where W x (t) = e − iHt W x e iHt . Using chaotic spin-chain numerics and gauge/gravity duality, we show that a single precursor fills a spatial region that grows linearly in t. In a lattice system, products of such operators can be represented using tensor networks. In gauge/gravity duality, they are related to Einstein-Rosen bridges supported by localized shock waves. We find a geometrical correspondence between these two descriptions, generalizing earlier work in the spatially homogeneous case.

We study chaotic dynamics in two-dimensional conformal field theory through out-of-time-order thermal correlators of the form ⟨W(t)VW(t)V⟩. We reproduce holographic calculations similar to those of Shenker and Stanford, by studying the large c Virasoro identity conformal block. The contribution of this block to the above correlation function begins to decrease exponentially after a delay of ~t_{*}-(β/2π)logβ^{2}E_{w}E_{v}, where t_{*} is the fast scrambling time (β/2π)logc and E_{w},E_{v} are the energy scales of the W,V operators.