Neural Field Discovery Disentangles Equivariance in Interacting Dynamical Systems

Systems of interacting objects often evolve under the influence of underlying field effects that govern their dynamics, \emph{e.g.} electromagnetic fields in physics, or map topologies and traffic rules in traffic scenes. While the interactions between objects depend on local information, the underlying fields depend on global states. Pedestrians and vehicles in traffic scenes, for example, follow different traffic rules and social norms depending on their absolute geolocation. The entanglement of global and local effects makes recently popularized equivariant networks inapplicable, since they fail to capture global information. To address this, in this work, we propose to \emph{disentangle} local object interactions --which are equivariant to global roto-translations and depend on relative positions and orientations-- from external global field effects --which depend on absolute positions and orientations. We theorize the presence of latent fields, which we aim to discover \emph{without} directly observing them, but infer them instead from the dynamics alone. We propose neural fields to learn the latent fields, and model the interactions with equivariant graph networks operating in local coordinate frames. We combine the two components in a graph network that transforms field effects in local frames and operates solely there. Our experiments show that we can accurately discover the underlying fields in charged particles settings, traffic scenes, and gravitational n-body problems, and effectively use them to learn the system and forecast future trajectories.