The foundations of statistical physics: entropy, irreversibility, and inference

Statistical physics aims to describe properties of macroscale systems in terms of distributions of their microscale agents. Its central tool is the maximization of entropy, a variational principle. We review the history of this principle, first considered as a law of nature, more recently as a procedure for inference in model-making. And while equilibria (EQ) have long been grounded in the principle of Maximum Entropy (MaxEnt), until recently no equally foundational generative principle has been known for non-equilibria (NEQ). We review evidence that the variational principle for NEQ is Maximum Caliber. It entails maximizing \textit{path entropies}, not \textit{state entropies}. We also describe the role of entropy in characterizing irreversibility, and describe the relationship between MaxCal and other prominent approaches to NEQ physics, including Stochastic Thermodynamics (ST), Large Deviations Theory (LDT), Macroscopic Fluctuation Theory (MFT), and non-extensive entropies.