Bio-Inspired Artificial Neural Networks Based on Predictive Coding [lecture Notes]

Backpropagation (BP) of errors is currently the backbone training algorithm for artificial neural networks (ANNs). It works by updating the network weights through gradient descent to minimize the value of a loss function, which represents the mismatch between the network’s prediction and the desired output. BP relies on the chain rule from calculus to propagate the loss gradient backward through the network’s hierarchy, allowing each weight to be efficiently and precisely updated based on its contribution to the output error. However, this process constrains the weight updates at every layer to rely on a global error signal generated at the extreme of the hierarchy. By contrast, the Hebbian model of synaptic plasticity in the brain states that weight updates should be local, determined only by the activity of presynaptic and postsynaptic neurons. According to Hebb’s model, it is therefore unlikely that biological brains directly implement BP. Recently, an alternative algorithm for training ANNs called predictive coding (PC) is gaining interest, appearing as a more biologically plausible alternative that updates the network weights using only local information. Originating from Elias’s 1950s work on signal compression [1], PC was later proposed in neuroscience as a model of the visual cortex by Rao and Ballard [2]. Successively, Friston formalized it under the free energy principle (FEP) [3], [4], grounding PC within the frameworks of Bayesian inference and dynamical systems. PC weight updates rely only on presynaptic and postsynaptic information, eliminating BP’s dependence on a global error signal. Moreover, it theoretically provides features beyond those of standard BP, such as the ability to automatically scale each gradient based on the associated uncertainty. Despite these advantages, PC still faces several open challenges: iterative error minimization can slow training compared to BP, and scaling to very deep architectures remains difficult [5]. At the same time, its local learning rules and modular structure offer promising opportunities, including highly parallelizable hardware implementations, biologically plausible learning in ANNs, and new connections between artificial intelligence (AI) and neuroscience. These aspects make PC an active area of research, with both theoretical and practical questions yet to be explored. This “Lecture Notes” column offers a novel, tutorial-style introduction to PC, focusing on its formulation, derivation, and connections to well-established optimization and signal processing algorithms, such as BP and the Kalman filter (KF). It aims to provide accessible support to the existing literature, guiding readers from the mathematical foundations underlying PC to its practical implementation, including computational examples in Python using the PyTorch framework.