AI帮你理解科学

AI 生成解读视频

AI抽取解析论文重点内容自动生成视频


pub
生成解读视频

AI 溯源

AI解析本论文相关学术脉络


Master Reading Tree
生成 溯源树

AI 精读

AI抽取本论文的概要总结


微博一下
We model system dynamics through a neural ordinary differential equation and draw the latent initial states for each object simultaneously through a novel encoder that is able to capture the interaction among objects

Learning Continuous System Dynamics fromIrregularly-Sampled Partial Observations

NIPS 2020, (2020)

被引用0|浏览49
下载 PDF 全文
引用
微博一下

摘要

Many real-world systems, such as moving planets, can be considered as multi-agent dynamic systems, where objects interact with each other and co-evolve along with the time. Such dynamics is usually difficult to capture, and understanding and predicting the dynamics based on observed trajectories of objects become a critical research probl...更多

代码

数据

0
简介
  • Learning system dynamics is a crucial task in a variety of domains, such as planning and control in robotics [1], predicting future movements of planets in physics [2], etc.
  • With the rapid development of deep learning techniques, researchers have started building neural-based simulators, aiming to approximate complex system interactions with neural networks [1, 3, 2, 4, 5] which can be learned automatically from data
  • Existing models, such as Interaction Networks (IN) [3], usually decompose the system into distinct objects and relations and learn to reason about the consequences of their interactions and dynamics based on graph neural network (GNN).
重点内容
  • Learning system dynamics is a crucial task in a variety of domains, such as planning and control in robotics [1], predicting future movements of planets in physics [2], etc
  • To learn a structural representation for each observation, we propose a temporal-aware graph neural network characterized by the information propagation equation in Eqn 4, where hlt−1, hls−1 are the representations of target and source node from layer l − 1 respectively
  • By comparing model variants for temporal self-attention module, we notice that taking the first observation as sequence representation produces high reconstruction error, which is expected as the first observable time for each sequence may not be the same so the inferred latent initial states are not aligned
  • We propose LG-Ordinary differential equations (ODE) for learning continuous multi-agent system dynamics from irregularly-sampled partial observations
  • We model system dynamics through a neural ordinary differential equation and draw the latent initial states for each object simultaneously through a novel encoder that is able to capture the interaction among objects
  • We achieve state-of-the-art performance in both interpolating missing values and extrapolating future trajectories
  • An limitation of current model is that we assume the underlying interaction graph is fixed over time
方法
  • The authors present Latent Graph ODE (LG-ODE) for learning continuous multi-agent system dynamics.
  • Following the structure of VAE, LG-ODE consists of three parts that are trained jointly: 1.) An encoder that infers the latent initial states of all objects simultaneously given their partiallyobserved trajectories; 2.) a generative model defined by an ODE function that learns the latent dynamics given the sampled initial states.
  • Let Zt ∈ RN×d denotes the latent state matrix of all N objects at time t.
  • Instead of encoding temporal pattern for each observation sequence oi
结果
  • Set up
  • In this task, the authors condition on a subset of observations (40%, 60%, 80%) from time (t0, tN ) and aim to reconstruct the full trajectories in the same time range.
  • Latent-ODE performs well on encoding single timeseries but fails to consider the interaction among objects, resulting in its poor performance in the multi-agent dynamic system setting.
  • Experiment results on model variants suggest that distinguishing the importance of nodes w.r.t time and incorporating temporal information via learnable positional encoding would benefit model performance.
  • The authors experiment on conditioning only on a subset of observations in the first half and run the encoder (a) Groundtruth
结论
  • Discussion and Conclusion

    In this paper, the authors propose LG-ODE for learning continuous multi-agent system dynamics from irregularly-sampled partial observations.
  • The authors model system dynamics through a neural ordinary differential equation and draw the latent initial states for each object simultaneously through a novel encoder that is able to capture the interaction among objects.
  • The joint learning of initial states captures interaction among objects but can benefit the learning when an object only has few observations.
  • An limitation of current model is that the authors assume the underlying interaction graph is fixed over time.
  • The authors plan to learn the system dynamics when the underlying interaction graph is evolving
总结
  • Introduction:

    Learning system dynamics is a crucial task in a variety of domains, such as planning and control in robotics [1], predicting future movements of planets in physics [2], etc.
  • With the rapid development of deep learning techniques, researchers have started building neural-based simulators, aiming to approximate complex system interactions with neural networks [1, 3, 2, 4, 5] which can be learned automatically from data
  • Existing models, such as Interaction Networks (IN) [3], usually decompose the system into distinct objects and relations and learn to reason about the consequences of their interactions and dynamics based on graph neural network (GNN).
  • Objectives:

    The authors assume there exists a latent generative continuous-time dynamic system, which the authors aim to uncover.
  • The authors' goal is to learn latent representations zti ∈ Rd for each object at any given time, and utilize it to reconstruct missing observations and forecast trajectories in the future.
  • Such process can be decomposed into two steps: 1.) Dynamic Node Representation Learning, where the authors aim to learn an encoding function fupdate that outputs structural contextualized representation hti for each observation oti.
  • Such process can be decomposed into two steps: 1.) Dynamic Node Representation Learning, where the authors aim to learn an encoding function fupdate that outputs structural contextualized representation hti for each observation oti. 2.) Temporal Self-Attention, where the authors learn an function faggre that aggregates the structural observation representations into a fixed-dimensional sequence representation ui for each object. ui is utilized to approximate the posterior distribution for each latent initial state z0i
  • Methods:

    The authors present Latent Graph ODE (LG-ODE) for learning continuous multi-agent system dynamics.
  • Following the structure of VAE, LG-ODE consists of three parts that are trained jointly: 1.) An encoder that infers the latent initial states of all objects simultaneously given their partiallyobserved trajectories; 2.) a generative model defined by an ODE function that learns the latent dynamics given the sampled initial states.
  • Let Zt ∈ RN×d denotes the latent state matrix of all N objects at time t.
  • Instead of encoding temporal pattern for each observation sequence oi
  • Results:

    Set up
  • In this task, the authors condition on a subset of observations (40%, 60%, 80%) from time (t0, tN ) and aim to reconstruct the full trajectories in the same time range.
  • Latent-ODE performs well on encoding single timeseries but fails to consider the interaction among objects, resulting in its poor performance in the multi-agent dynamic system setting.
  • Experiment results on model variants suggest that distinguishing the importance of nodes w.r.t time and incorporating temporal information via learnable positional encoding would benefit model performance.
  • The authors experiment on conditioning only on a subset of observations in the first half and run the encoder (a) Groundtruth
  • Conclusion:

    Discussion and Conclusion

    In this paper, the authors propose LG-ODE for learning continuous multi-agent system dynamics from irregularly-sampled partial observations.
  • The authors model system dynamics through a neural ordinary differential equation and draw the latent initial states for each object simultaneously through a novel encoder that is able to capture the interaction among objects.
  • The joint learning of initial states captures interaction among objects but can benefit the learning when an object only has few observations.
  • An limitation of current model is that the authors assume the underlying interaction graph is fixed over time.
  • The authors plan to learn the system dynamics when the underlying interaction graph is evolving
表格
  • Table1: Mean Squared Error(MSE) ×10−2 on Interpolation task
  • Table2: Mean Squared Error(MSE) ×10−2 on Extrapolation task
Download tables as Excel
相关工作
  • Neural Physical Simulator. Existing works have developed various neural-based physical simulators that learn the system dynamics from data [1, 3]. In particular, Kipf et al [2] and Battaglia et al [3] have explored learning a simulator by approximating pair-wise object interactions with graph neural networks. These approaches restrict themselves to learn a fixed-step state transition function that takes the system state at time t as input to predict the state at time t + 1. However, they can not be applied to the scenarios where system observations are irregularly sampled. Our model handles such issue by combining a neural ODE [8] to model continuous system dynamics and a temporal-aware graph neural network followed by a temporal self-attention module to estimate system initial states. Another issue lies in that they need to observe the full states of a system; but in reality, system states are often partially observed where number and set of observable objects vary over time. A recent work [1] tackled this issue where system is partially observed but observations are regularly sampled by learning the dynamics over a latent global representation for the system, which is for example an average over the sets of object states. However it cannot directly learn the dynamic state for each object. In our work, we design a dynamic model that explicitly operates on the latent dynamic representations over each object. This allows us to define object-centric dynamics, which can better capture system dynamics compared to the coarse global system representation.
基金
  • This work is partially supported by NSF III-1705169, NSF CAREER Award 1741634, NSF 1937599, DARPA HR00112090027, Okawa Foundation Grant,Amazon Research, NSF DBI-1565137, DGE1829071, IIS-2031187, NIH R35-HL135772, NEC Research Gift, and Verizon Media Faculty Research and Engagement Program
研究对象与分析
observations: 40
To generate irregularly-sampled partial observations, for each particle we sample the number of observations n from U(40, 52) and draw the n observations uniformly from the PDE steps to get the training trajectories. To evaluate extrapolation task, we additionally sample 40 observations following the same procedure from PDE steps [6000, 12000] for testing. The above sampling procedure is conducted independently for each object

observations: 40
Similar as simulated datasets, for each joint we sample the number of observations n from U(30, 42) and draw the n observations uniformly from first 50 frames for training trajectories. For testing, we additionally sampled 40 observations from frames [51, 99]. We split the different walking trials into non-overlapping training (15 trials) and test sets (7 trials)

引用论文
  • Yunzhu Li, Jiajun Wu, Jun-Yan Zhu, Joshua B Tenenbaum, Antonio Torralba, and Russ Tedrake. Propagation networks for model-based control under partial observation. In ICRA, 2019.
    Google ScholarLocate open access versionFindings
  • Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, and Richard Zemel. Neural relational inference for interacting systems. arXiv preprint arXiv:1802.04687, 2018.
    Findings
  • Peter Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, and koray kavukcuoglu. Interaction networks for learning about objects, relations and physics. In Advances in Neural Information Processing Systems 29, pages 4502–4510. 2016.
    Google ScholarLocate open access versionFindings
  • Michael B Chang, Tomer Ullman, Antonio Torralba, and Joshua B Tenenbaum. A compositional object-based approach to learning physical dynamics. ICLR, 2016.
    Google ScholarLocate open access versionFindings
  • Sainbayar Sukhbaatar, arthur szlam, and Rob Fergus. Learning multiagent communication with backpropagation. In Advances in Neural Information Processing Systems 29, pages 2244–2252. 2016.
    Google ScholarLocate open access versionFindings
  • Xianfeng Tang, Huaxiu Yao, Yiwei Sun, Charu Aggarwal, Prasenjit Mitra, and Suhang Wang. Joint modeling of local and global temporal dynamics for multivariate time series forecasting with missing values. In AAAI. 2020.
    Google ScholarFindings
  • Yulia Rubanova, Ricky T. Q. Chen, and David K Duvenaud. Latent ordinary differential equations for irregularly-sampled time series. In Advances in Neural Information Processing Systems 32, pages 5320–5330. 2019.
    Google ScholarLocate open access versionFindings
  • Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems 31, pages 6571– 6583. 2018.
    Google ScholarLocate open access versionFindings
  • Simo Särkkä Michael Schober and Philipp Hennig. A probabilistic model for the numerical solution of initial value problems. In Statistics and Computing, pages 99–122. 2019.
    Google ScholarLocate open access versionFindings
  • Cagatay Yildiz, Markus Heinonen, and Harri Lahdesmaki. Ode2vae: Deep generative second order odes with bayesian neural networks. In Advances in Neural Information Processing Systems 32, pages 13412–13421. 2019.
    Google ScholarLocate open access versionFindings
  • Adam Santoro, David Raposo, David G Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap. A simple neural network module for relational reasoning. In Advances in Neural Information Processing Systems 30, pages 4967–4976. 2017.
    Google ScholarLocate open access versionFindings
  • Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014.
    Google ScholarLocate open access versionFindings
  • Aravind Sankar, Yanhong Wu, Liang Gou, Wei Zhang, and Hao Yang. Dysat: Deep neural representation learning on dynamic graphs via self-attention networks. In Proceedings of the 13th International Conference on Web Search and Data Mining, pages 519–527, 2020.
    Google ScholarLocate open access versionFindings
  • L. Zhu, D. Guo, J. Yin, G. V. Steeg, and A. Galstyan. Scalable temporal latent space inference for link prediction in dynamic social networks. IEEE Transactions on Knowledge and Data Engineering, 28(10):2765–2777, 2016.
    Google ScholarLocate open access versionFindings
  • Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems 30, pages 5998–6008. 2017.
    Google ScholarLocate open access versionFindings
  • CMU. Carnegie-mellon motion capture database. 2003. URL http://mocap.cs.cmu.edu.
    Findings
  • L. Gong and Q. Cheng. Exploiting edge features for graph neural networks. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9203–9211, 2019.
    Google ScholarLocate open access versionFindings
  • Wei Cao, Dong Wang, Jian Li, Hao Zhou, Lei Li, and Yitan Li. Brits: Bidirectional recurrent imputation for time series. In Advances in Neural Information Processing Systems 31, pages 6775–6785. 2018.
    Google ScholarLocate open access versionFindings
  • Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan. Liu. Recurrent neural networks for multivariate time series with missing values. In Scientific Reports, page 6085, 2018.
    Google ScholarLocate open access versionFindings
  • Ehsan Hajiramezanali, Arman Hasanzadeh, Krishna Narayanan, Nick Duffield, Mingyuan Zhou, and Xiaoning Qian. Variational graph recurrent neural networks. In Advances in Neural Information Processing Systems 32, pages 10701–10711. 2019.
    Google ScholarLocate open access versionFindings
  • Ziniu Hu, Yuxiao Dong, Kuansan Wang, and Yizhou Sun. Heterogeneous graph transformer. In Proceedings of the 2020 World Wide Web Conference, 2020.
    Google ScholarLocate open access versionFindings
您的评分 :
0

 

标签
评论
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科