As the second inverse problem in structural dynamics, dynamic load identification is highly dependent on the system’s intrinsic properties. For distributed loads, establishing an accurate mapping between structural responses and the underlying dynamic excitations remains particularly challenging, and the ill-posed nature of the problem further amplifies measurement noise, leading to significant identification errors. To overcome these difficulties, this study proposes a novel distributed dynamic load identification framework based on a Transformer architecture that directly learns the inverse dynamic relationship without requiring explicit system parameter estimation. Specifically, Legendre orthogonal polynomial decomposition is first employed to transform the load identification task into the estimation of a finite set of orthogonal polynomial coefficients. Building upon this framework, innovative architectural optimizations are introduced by embedding physical constraints into attention computation and linear prediction, leveraging the temporal causality of dynamic responses. These enhancements improve model interpretability and substantially reduce training difficulty. Numerical simulations demonstrate that the proposed method can accurately identify sinusoidal, impact, and random loads under various noise levels. Furthermore, a distributed load identification experiment on a cantilever beam is carried out, validating the practical applicability of the approach. Finally, the selection of model hyperparameters is discussed based on fitting and generalization performance, and a comparative study with traditional dynamic calibration methods was conducted in an experimental setting, further demonstrating the superior accuracy, noise robustness, and practical reliability of the proposed framework.