Recent experiments have shown that training trajectories of multiple deep neural networks with different architectures, optimization algorithms, hyper-parameter settings, and regularization methods evolve on a remarkably low-dimensional "hyper-ribbon-like" manifold in the space of probability distributions. Inspired by the similarities in the training trajectories of deep networks and linear networks, we analytically characterize this phenomenon for the latter. We show, using tools in dynamical systems theory, that the geometry of this low-dimensional manifold is controlled by (i) the decay rate of the eigenvalues of the input correlation matrix of the training data, (ii) the relative scale of the ground-truth output to the weights at the beginning of training, and (iii) the number of steps of gradient descent. By analytically computing and bounding the contributions of these quantities, we characterize phase boundaries of the region where hyper-ribbons are to be expected. We also extend our analysis to kernel machines and linear models that are trained with stochastic gradient descent.

Recently, we proposed a universal scaling framework that shows shear thickening in dense suspensions is governed by the crossover between two critical points: one associated with frictionless isotropic jamming and a second corresponding to frictional shear jamming. Here, we show that orthogonal perturbations to the flows, an effective method for tuning shear thickening, can also be folded into this universal scaling framework. Specifically we show that the effect of adding orthogonal shear perturbations can be incorporated into the scaling variable via a multiplicative function, determined through our measurements, to achieve collapse of the entire thickening and dethickening dataset onto a single universal curve. We then show that this universal scaling framework can be used to control the phase behavior in thickening and jamming suspensions.

Tuning shear thickening behavior is a longstanding problem in the field of dense suspensions. Acoustic perturbations offer a convenient way to control shear thickening in real time, opening the door to a new class of smart materials. However, complete control over shear thickening requires a quantitative description for how suspension viscosity varies under acoustic perturbation. Here, we achieve this goal by experimentally probing suspensions with acoustic perturbations and incorporating their effect on the suspension viscosity into a universal scaling framework where the viscosity is described by a scaling function, which captures a crossover from the frictionless jamming critical point to a frictional shear jamming critical point. Our analysis reveals that the effect of acoustic perturbations may be explained by the introduction of an effective interparticle repulsion whose magnitude is roughly equal to the acoustic energy density. Furthermore, we demonstrate how this scaling framework may be leveraged to produce explicit predictions for the viscosity of a dense suspension under acoustic perturbation. Our results demonstrate the utility of the scaling framework for experimentally manipulating shear thickening systems.

We study how the rigidity transition in a triangular lattice changes as a function of anisotropy by preferentially filling bonds on the lattice in one direction. We discover that the onset of rigidity in anisotropic spring networks on a regular triangular lattice arises in at least two steps, reminiscent of the two-step melting transition in two dimensional crystals. In particular, our simulations demonstrate that the percolation of stress-supporting bonds happens at different critical volume fractions along different directions. By examining each independent component of the elasticity tensor, we determine universal exponents and develop universal scaling functions to analyze isotropic rigidity percolation as a multicritical point. Our crossover scaling approach is applicable to anisotropic biological materials (e.g. cellular cytoskeletons, extracellular networks of tissues like tendons), and extensions to this analysis are important for the strain stiffening of these materials.

Universal scaling laws only apply asymptotically near critical phase transitions. We propose a general scheme, based on normal form theory of renormalization group flows, for incorporating corrections to scaling that quantitatively describe the entire neighboring phases. Expanding Onsager's exact solution of the 2D Ising model about the critical point, we identify a special coordinate with radius of convergence covering the entire physical temperature range, 0<T<∞. Without an exact solution, we demonstrate that using solely the critical singularity with low- and high-temperature expansions leads to exponentially converging approximations across all temperatures for both the 2D and 3D Ising free energies and the 3D magnetization. We discuss challenges and opportunities for future work.

We cast a nonzero-temperature analysis of the jamming transition into the framework of a scaling ansatz. We show that four distinct regimes for scaling exponents of thermodynamic derivatives of the free energy such as pressure, bulk and shear moduli, can be consolidated by introducing a universal scaling function with two branches. Both the original analysis and the scaling theory assume that the system always resides in a single basis in the energy landscape. The two branches are separated by a line $T^*(\Delta \phi)$ in the $T-\Delta \phi$ plane, where $\Delta \phi=\phi-\phi_c^\Lambda$ is the deviation of the packing fraction from its critical, jamming value, $\phi_c^\Lambda$, for that basin. The branch for $TT^*(\Delta \phi)$ reproduces exponents observed for thermal hard spheres. In contrast to the usual scenario for critical phenomena, the two branches are characterized by different exponents. We suggest that this unusual feature can be resolved by the existence of a dangerous irrelevant variable $u$, which can appear to modify exponents if the leading $u=0$ term is sufficiently small in the regime described by one of the two branches of the scaling function.

Systems driven far from equilibrium often retain structural memories of their processing history. This memory has, in some cases, been shown to dramatically alter the material response. For example, work hardening in crystalline metals can alter the hardness, yield strength, and tensile strength to prevent catastrophic failure. Whether memory of processing history can be similarly exploited in flowing systems, where significantly larger changes in structure should be possible, remains poorly understood. Here, we demonstrate a promising route to embedding such useful memories. We build on work showing that exposing a sheared dense suspension to acoustic perturbations of different power allows for dramatically tuning the sheared suspension viscosity and underlying structure. We find that, for sufficiently dense suspensions, upon removing the acoustic perturbations, the suspension shear jams with shear stress contributions from the maximum compressive and maximum extensive axes that reflect or "remember" the acoustic training. Because the contributions from these two orthogonal axes to the total shear stress are antagonistic, it is possible to tune the resulting suspension response in surprising ways. For example, we show that differently trained sheared suspensions exhibit (1) different susceptibility to the same acoustic perturbation, (2) orders of magnitude changes in their instantaneous viscosities upon shear reversal, and (3) even a shear stress that increases in magnitude upon shear cessation. We work through these examples to explain the underlying mechanisms governing each behavior. Then, to illustrate the power of this approach for controlling suspension properties, we demonstrate that flowing states well below the shear jamming threshold can be shear jammed via acoustic training. Collectively, our work paves the way for using acoustically induced memory in dense suspensions to generate rapidly and widely tunable materials.

Circular DNA found in the cell is actively regulated to an underwound state, with their superhelical density close to σ ~ - 0 . 06 . While this underwound state is essential to life, how it impacts the torsional mechanical properties of DNA is not fully understood. In this work, we performed simulations to understand the torsional mechanics of circular DNA and validated our results with single-molecule measurements and analytical theory. We found that the torque generated at σ ~ - 0 . 06 is near but slightly below that required to melt DNA, significantly decreasing the energy barrier for proteins that interact with melted DNA. Furthermore, supercoiled circular DNA experiences force (tension) and torque that are equally distributed through the DNA contour. We have also extended a previous analytical framework to show how the plectonemic twist persistence length depends on the intrinsic bending persistence length and twist persistence length. Our work establishes a framework for understanding DNA supercoiling and torsional dynamics of circular DNA.

Natural systems with emergent behaviors often organize along low-dimensional subsets of high-dimensional spaces. For example, despite the tens of thousands of genes in the human genome, the principled study of genomics is fruitful because biological processes rely on coordinated organization that results in lower dimensional phenotypes. To uncover this organization, many nonlinear dimensionality reduction techniques have successfully embedded high-dimensional data into low-dimensional spaces by preserving local similarities between data points. However, the nonlinearities in these methods allow for too much curvature to preserve general trends across multiple non-neighboring data clusters, thereby limiting their interpretability and generalizability to out-of-distribution data. Here, we address both of these limitations by regularizing the curvature of manifolds generated by variational autoencoders, a process we coin ``$\Gamma$-VAE''. We demonstrate its utility using two example data sets: bulk RNA-seq from the The Cancer Genome Atlas (TCGA) and the Genotype Tissue Expression (GTEx); and single cell RNA-seq from a lineage tracing experiment in hematopoietic stem cell differentiation. We find that the resulting regularized manifolds identify mesoscale structure associated with different cancer cell types, and accurately re-embed tissues from completely unseen, out-of distribution cancers as if they were originally trained on them. Finally, we show that preserving long-range relationships to differentiated cells separates undifferentiated cells -- which have not yet specialized -- according to their eventual fate. Broadly, we anticipate that regularizing the curvature of generative models will enable more consistent, predictive, and generalizable models in any high-dimensional system with emergent low-dimensional behavior.

Our community has a deep and sophisticated understanding of phase transitions and their universal scaling functions. We outline and advocate an ambitious program to use this understanding as an anchor for describing the surrounding phases. We explain how to use normal form theory to write universal scaling functions in systems where the renormalization-group flows cannot be linearized. We use the 2d Ising model to demonstrate how to calculate high-precision implementations of universal scaling functions, and how to extend them into a complete description of the surrounding phases. We discuss prospects and challenges involved into extending these early successes to the many other systems where the RG has successfully described emergent scale invariance, making them invaluable tools for engineers, biologists, and social scientists studying complex systems.

We develop information geometric techniques to understand the representations learned by deep networks when they are trained on different tasks using supervised, meta-, semi-supervised and contrastive learning. We shed light on the following phenomena that relate to the structure of the space of tasks: (1) the manifold of probabilistic models trained on different tasks using different representation learning methods is effectively low-dimensional; (2) supervised learning on one task results in a surprising amount of progress even on seemingly dissimilar tasks; progress on other tasks is larger if the training task has diverse classes; (3) the structure of the space of tasks indicated by our analysis is consistent with parts of the Wordnet phylogenetic tree; (4) episodic meta-learning algorithms and supervised learning traverse different trajectories during training but they fit similar models eventually; (5) contrastive and semi-supervised learning methods traverse trajectories similar to those of supervised learning. We use classification tasks constructed from the CIFAR-10 and Imagenet datasets to study these phenomena.

We propose a design paradigm for multistate machines where transitions from one state to another are organized by bifurcations of multiple equilibria of the energy landscape describing the collective interactions of the machine components. This design paradigm is attractive since, near bifurcations, small variations in a few control parameters can result in large changes to the system's state providing an emergent lever mechanism. Further, the topological configuration of transitions between states near such bifurcations ensures robust operation, making the machine less sensitive to fabrication errors and noise. To design such machines, we develop and implement a new efficient algorithm that searches for interactions between the machine components that give rise to energy landscapes with these bifurcation structures. We demonstrate a proof of concept for this approach by designing magnetoelastic machines whose motions are primarily guided by their magnetic energy landscapes and show that by operating near bifurcations we can achieve multiple transition pathways between states. This proof of concept demonstration illustrates the power of this approach, which could be especially useful for soft robotics and at the microscale where typical macroscale designs are difficult to implement.

The strange metallic regime across a number of high-temperature superconducting materials presents numerous challenges to the classic theory of Fermi liquid metals. Recent measurements of the dynamical charge response of strange metals, including optimally doped cuprates, have revealed a broad, featureless continuum of excitations, extending over much of the Brillouin zone. The collective density oscillations of this strange metal decay into the continuum in a manner that is at odds with the expectations of Fermi liquid theory. Inspired by these observations, we investigate the phenomenology of bosonic collective modes and the particle-hole excitations in a class of strange metals by making an analogy to the phonons of classical lattices falling apart across an unconventional jamming-like transition associated with the onset of rigidity. By making comparisons to the experimentally measured dynamical response functions, we reproduce many of the qualitative features using the above framework. We conjecture that the dynamics of electronic charge density over an intermediate range of energy scales in a class of strongly correlated metals can be at the brink of a jamming-like transition.

Spin Glass Theory and Far Beyond, pp. 277-305 (2023) No AccessChapter 15: Emergent Dynamics in Glasses and Disordered Systems: Correlations and AvalanchesAnnette Zippelius, Matthias Fuchs, Alberto Rosso, James P. Sethna, and Matthieu WyartAnnette ZippeliusUniversity Göttingen, D-37077 Göttingen, Germany, Matthias FuchsUniversity of Konstanz, D-78457 Konstanz, Germany, Alberto RossoUniversité Paris Saclay, CNRS,LPTMS, 91405, Orsay, France, James P. SethnaLASSP, Cornell University, Ithaca, NY 14853, USA, and Matthieu WyartInstitute of Physics, École Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerlandhttps://doi.org/10.1142/9789811273926_0015Cited by:0 PreviousNext AboutSectionsPDF/EPUB ToolsAdd to favoritesDownload CitationsTrack CitationsRecommend to Library ShareShare onFacebookTwitterLinked InRedditEmail Abstract: This chapter reviews some emergent properties of the dynamics of glassy systems, and more generally of a wide class of disordered systems. In a first contribution (Sec. 15.1), Zippelius and Fuchs review the emergence of stress propagation upon cooling a glass-forming material. They show that emergent solid elasticity leads to long-ranged stress correlations both in the supercooled liquid and glass phases, having a non-trivial quadrupolar structure of the Eshelby type. In a second contribution (Sec. 15.2), Rosso, Sethna and Wyart discuss how such long-range stress correlations lead to frustrated interactions between local excitations and the emergence of avalanches in glasses, focusing in particular on the rigid phase at negligible temperature. More generally, they draw an analogy with other avalanche phenomena appearing in a variety of disordered systems, from epidemic models to the depinning transition of elastic interfaces to the jamming transition of granular materials. FiguresReferencesRelatedDetails Recommended Spin Glass Theory and Far BeyondMetrics History PDF download

Nearly, all dense suspensions undergo dramatic and abrupt thickening transitions in their flow behavior when sheared at high stresses. Such transitions occur when the dominant interactions between the suspended particles shift from hydrodynamic to frictional. Here, we interpret abrupt shear thickening as a precursor to a rigidity transition and give a complete theory of the viscosity in terms of a universal crossover scaling function from the frictionless jamming point to a rigidity transition associated with friction, anisotropy, and shear. Strikingly, we find experimentally that for two different systems—cornstarch in glycerol and silica spheres in glycerol—the viscosity can be collapsed onto a single universal curve over a wide range of stresses and volume fractions. The collapse reveals two separate scaling regimes due to a crossover between frictionless isotropic jamming and frictional shear jamming, with different critical exponents. The material-specific behavior due to the microscale particle interactions is incorporated into a scaling variable governing the proximity to shear jamming, that depends on both stress and volume fraction. This reformulation opens the door to importing the vast theoretical machinery developed to understand equilibrium critical phenomena to elucidate fundamental physical aspects of the shear thickening transition.

We develop a technique to analyze representations learned by deep networks when they are trained on different tasks using supervised, multi-task, meta- and contrastive learning. We develop a technique to visualize such representations using an isometric embedding of the space of probabilistic models into a lower-dimensional space, i.e., one that preserves pairwise distances. We discover the following surprising phenomena that shed light upon the structure in the space of learning tasks: (1) the manifold of probabilistic models trained on different tasks using different representation learning methods is effectively low-dimensional; (2) supervised learning on one task results in a surprising amount of progress on seemingly dissimilar tasks; progress on other tasks is larger if the training task has diverse classes; (3) the structure of the space of tasks indicated by our visualization technique is consistent with parts of the Wordnet phylogenetic tree; (4) fine-tuning a model upon a sub-task does not change the representation much if the model was trained for a large number of epochs; (5) episodic meta-learning algorithms fit similar models eventually as that of supervised learning, even if the two traverse different trajectories during training; (6) contrastive learning methods trained on different datasets learn similar representations. We use classification tasks constructed from the CIFAR-10 and Imagenet datasets to study these phenomena.

Recent experiments on superconducting cavities have found that under large radio-frequency (RF) electromagnetic fields the quality factor can improve with increasing field amplitude, a so-called "anti-Q slope." Linear theories of dissipation break down under these extreme conditions and are unable to explain this behavior. We numerically solve the Bogoliubov-de Gennes equations at the surface of a superconductor in a parallel AC magnetic field, finding that at large fields there are quasiparticle surface states with energies below the bulk value of the superconducting gap. As the field oscillates, such states emerge and disappear with every cycle. We consider the dissipation resulting from inelastic quasiparticle-phonon scattering into these states and investigate the ability of this mechanism to explain features of the experimental observations, including the field dependence of the quality factor. We find that this mechanism is likely not the dominant source of dissipation and does not produce an anti-Q slope by itself; however, we demonstrate in a modified two-fluid model how these bound states can play a role in producing an anti-Q slope.

Complex models in physics, biology, economics, and engineering are often sloppy , meaning that the model parameters are not well determined by the model predictions for collective behavior. Many parameter combinations can vary over decades without significant changes in the predictions. This review uses information geometry to explore sloppiness and its deep relation to emergent theories. We introduce the model manifold of predictions, whose coordinates are the model parameters. Its hyperribbon structure explains why only a few parameter combinations matter for the behavior. We review recent rigorous results that connect the hierarchy of hyperribbon widths to approximation theory, and to the smoothness of model predictions under changes of the control variables. We discuss recent geodesic methods to find simpler models on nearby boundaries of the model manifold—emergent theories with fewer parameters that explain the behavior equally well. We discuss a Bayesian prior which optimizes the mutual information between model parameters and experimental data, naturally favoring points on the emergent boundary theories and thus simpler models. We introduce a ‘projected maximum likelihood’ prior that efficiently approximates this optimal prior, and contrast both to the poor behavior of the traditional Jeffreys prior. We discuss the way the renormalization group coarse-graining in statistical mechanics introduces a flow of the model manifold, and connect stiff and sloppy directions along the model manifold with relevant and irrelevant eigendirections of the renormalization group. Finally, we discuss recently developed ‘intensive’ embedding methods, allowing one to visualize the predictions of arbitrary probabilistic models as low-dimensional projections of an isometric embedding, and illustrate our method by generating the model manifold of the Ising model.

The onset of rigidity in interacting liquids, as they undergo a transition to a disordered solid, is associated with a rearrangement of the low-frequency vibrational spectrum. In this Letter, we derive scaling forms for the singular dynamical response of disordered viscoelastic networks near both jamming and rigidity percolation. Using effective-medium theory, we extract critical exponents, invariant scaling combinations, and analytical formulas for universal scaling functions near these transitions. Our scaling forms describe the behavior in space and time near the various onsets of rigidity, for rigid and floppy phases and the crossover region, including diverging length scales and timescales at the transitions.

In this chapter, we discuss avalanches in glasses and disordered systems, and the macroscopic dynamical behavior that they mediate. We briefly review three classes of systems where avalanches are observed: depinning transition of disordered interfaces, yielding of amorphous materials, and the jamming transition. Without extensive formalism, we discuss results gleaned from theoretical approaches – mean-field theory, scaling and exponent relations, the renormalization group, and a few results from replica theory. We focus both on the remarkably sophisticated physics of avalanches and on relatively new approaches to the macroscopic flow behavior exhibited past the depinning/yielding transition.