Training Neural Networks from an Ergodic Perspective

In this research, we viewed the weights of a neural network as points in a metric space. We proposed that the process of training a neural network can be seen as an iterated function system on this space. Additionally, we found that the most effective training method involves starting with initial weights that are very close to the minimum error. We also provided an ergodic characterization of this efficient training. Our findings suggest that there is potential for further optimization advancements through numerical experimentation and the study of dynamical systems theory.