## AI helps you reading Science

## AI Insight

AI extracts a summary of this paper

Weibo:

# Learning Neural Generative Dynamics for Molecular Conformation Generation

ICLR, (2021)

EI

Keywords

Abstract

We study how to generate molecule conformations (\textit{i.e.}, 3D structures) from a molecular graph. Traditional methods, such as molecular dynamics, sample conformations via computationally expensive simulations. Recently, machine learning methods have shown great potential by training on a large collection of conformation data. Challe...More

Code:

Data:

Introduction

- The authors have witnessed the success of graph-based representations for molecular modeling in a variety of tasks such as property prediction (Gilmer et al, 2017) and molecule generation (You et al, 2018; Shi et al, 2020).
- A more natural and intrinsic representation of a molecule is its 3D structure, commonly known as the molecular geometry or conformation, which represents each atom by its 3D coordinate.
- Generating valid and stable conformations of a given molecule remains very challenging.
- Such structures are determined by expensive and time-consuming crystallography.
- Computational approaches based on Markov chain Monte Carlo (MCMC) or molecular dynamics (MD) (De Vivo et al, 2016) are computationally expensive, especially for large molecules (Ballard et al, 2015)

Highlights

- We have witnessed the success of graph-based representations for molecular modeling in a variety of tasks such as property prediction (Gilmer et al, 2017) and molecule generation (You et al, 2018; Shi et al, 2020)
- Numerical evaluations show that our proposed framework consistently outperforms the previous state-of-the-art (GraphDG) on both conformation generation and distance modeling tasks, with a clear margin
- We propose to take the pre-trained Conditional Graph Continuous Flow (CGCF) to serve as a strong noise distribution, leading to the following discriminative learning objective for the ETM1: Lnce(R, G; φ) = − Epdata log
- As a continuous normalizing flow (CNF)-based model, CGCF holds much the higher generative capacity for both diversity and quality compared than VAE approaches
- A meaningful observation is that though competitive over other neural models, the rule-based RDKit method occasionally shows better performance than our model, which indicates that RDKit can generate more realistic structures
- We propose a novel probabilistic framework for molecular conformation generation

Methods

- The authors first present a high-level description of the model.
- Learning a generative model on Cartesian coordinates heavily depends on the rotation and translation (Mansimov et al, 2019).
- In this paper the authors take the atomic pairwise distances as intermediate variables to generate Input Graph.
- G
AAAB8nicbVDLSgMxFM3UV62vqks3wSK4kDIjBXVXcKHLCvYB06Fk0kwbmkmG5I5Qhn6GGxeKuPVr3Pk3ZtpZaOuBwOGce8m5J0wEN+C6305pbX1jc6u8XdnZ3ds/qB4edYxKNWVtqoTSvZAYJrhkbeAgWC/RjMShYN1wcpv73SemDVfyEaYJC2IykjzilICV/H5MYEyJyO5mg2rNrbtz4FXiFaSGCrQG1a/+UNE0ZhKoIMb4nptAkBENnAo2q/RTwxJCJ2TEfEsliZkJsnnkGT6zyhBHStsnAc/V3xsZiY2ZxqGdzCOaZS8X//P8FKLrIOMySYFJuvgoSgUGhfP78ZBrRkFMLSFUc5sV0zHRhIJtqWJL8JZPXiWdy7rXqN88NGrNi6KOMjpBp+gceegKNdE9aqE2okihZ/SK3hxwXpx352MxWnKKnWP0B87nD3aukVM= p✓ (d|G ). - Predict distances for the input graph.
- Search 3D coordinates given the distances.

Results

- Tab. 1 shows that compared with the existing state-of-the-art baselines, the CGCF model can already achieve superior performance on all four metrics.
- ETM will slightly hurt the results
- This phenomenon is consistent with the observations for RDKit. Instead of generating unbiased samples from the underlying distribution, RDKit will only generate the stable ones with local minimal energy by involving the hand-designed molecular force field (Simm & Hernandez-Lobato, 2020).
- Once all bonded and non-bonded interactions, plus optional restraints, have been loaded into the MMFF energy expression, potential gradients of the system under study can be computed to minimize the energy

Conclusion

- The authors propose a novel probabilistic framework for molecular conformation generation.
- The authors' generative model combines the advantage of both flow-based and energy-based models, which is capable of modeling the complex multi-modal geometric distribution and highly branched atomic correlations.
- Experimental results show that the method outperforms all previous state-of-the-art baselines on the standard benchmarks.
- Future work includes applying the framework on much larger datasets and extending it to more challenging structures

- Table1: Comparison of different methods on the COV and MAT scores. Top 4 rows: deep generative models for molecular conformation generation. Bottom 5 rows: different methods that involve an additional rule-based force field to further optimize the generated structures
- Table2: Comparison of distances density modeling with different methods. We compare the marginal distribution of single (p(duv|G)), pair (p(duv, dij|G)) and all (p(d|G)) edges between C and O atoms. Molecular graphs G are taken from the test set of ISO17. We take two metrics into consideration: 1) median MMD between the ground truth and generated ones, and 2) mean ranking (1 to 3) based on the MMD metric
- Table3: Conformation Diversity. Mean and Std represent the corresponding mean and standard deviation of pairwise RMSD between the generated conformations per molecule
- Table4: Comparison of different methods on the JUNK scores. Top 4 rows: deep generative models for molecular conformation generation. Bottom 5 rows: different methods that involve an additional rule-based force field to further optimize the generated structures
- Table5: Mean Absolute Error (MAE) in energy properties between ground truth and generated conformations from different methods

Related work

- Conformation Generation. There have been results showing deep learning speeding up molecular dynamics simulation by learning efficient alternatives to quantum mechanics-based energy calculations (Schutt et al, 2017; Smith et al, 2017). However, though accelerated by neural networks, these approaches are still time-consuming due to the lengthy MCMC process. Recently, Gebauer et al (2019) and Hoffmann & Noe (2019) propose to directly generate 3D structures with deep generative models. However, these models can hardly capture graph- or bond-based structure, which is typically complex and highly branched. Some other works (Lemke & Peter, 2019; AlQuraishi, 2019; Ingraham et al, 2019; Noeet al., 2019; Senior et al, 2020) also focus on learning models to directly generate 3D structure, but focus on the protein folding problem. Unfortunately, proteins are linear structures while general molecules are highly branched, making these methods not naturally transferable to general molecular conformation generation tasks.

Funding

- Numerical evaluations show that our proposed framework consistently outperforms the previous state-of-the-art (GraphDG) on both conformation generation and distance modeling tasks, with a clear margin
- Experimental results show that our method outperforms all previous state-of-the-art baselines on the standard benchmarks

Study subjects and analysis

conformation-molecule pairs: 50000

This dataset is limited to 9 heavy atoms (29 total atoms), with small molecular mass and few rotatable bonds. We randomly draw 50000 conformation-molecule pairs from GEOM-QM9 to be the training set, and take another 17813 conformations covering 150 molecular graphs as the test set. GEOM-Drugs dataset consists of much larger drug molecules, up to a maximum of 181 atoms (91 heavy atoms)

conformationmolecule pairs: 50000

It also contains multiple conformations for each molecule, with a larger variance in structures, e.g., there are the 6.5 rotatable bonds in average. We randomly take 50000 conformationmolecule pairs from GEOM-Drugs as the training set, and another 9161 conformations (covering 100 molecular graphs) as the test split. ISO17 dataset is also built upon QM9 datasets, which consists of 197 molecules, each with 5000 conformations

Reference

- Mohammed AlQuraishi. End-to-end differentiable learning of protein structure. Cell systems, 8(4): 292–301, 2019.
- Simon Axelrod and Rafael Gomez-Bombarelli. Geom: Energy-annotated molecular conformations for property prediction and molecular generation. arXiv preprint arXiv:2006.05531, 2020.
- Andrew J Ballard, Stefano Martiniani, Jacob D Stevenson, Sandeep Somani, and David J Wales. Exploiting the potential energy landscape to sample free energy. Wiley Interdisciplinary Reviews: Computational Molecular Science, 5(3):273–289, 2015.
- Yoshua Bengio, Gregoire Mesnil, Yann Dauphin, and Salah Rifai. Better mixing via deep representations. In International conference on machine learning, pp. 552–560, 2013a.
- Yoshua Bengio, Li Yao, Guillaume Alain, and Pascal Vincent. Generalized denoising auto-encoders as generative models. In Advances in neural information processing systems, pp. 899–907, 2013b.
- Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In Advances in neural information processing systems, pp. 6571–6583, 2018.
- Gordon M Crippen, Timothy F Havel, et al. Distance geometry and molecular conformation, volume 74. Research Studies Press Taunton, 1988.
- Peter Dayan, Geoffrey E Hinton, Radford M Neal, and Richard S Zemel. The helmholtz machine. Neural computation, 7(5):889–904, 1995.
- Marco De Vivo, Matteo Masetti, Giovanni Bottegoni, and Andrea Cavalli. Role of molecular dynamics and related methods in drug discovery. Journal of medicinal chemistry, 59(9):4035–4061, 2016.
- Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent components estimation. arXiv preprint arXiv:1410.8516, 2014.
- Yilun Du and Igor Mordatch. Implicit generation and generalization in energy-based models. arXiv preprint arXiv:1903.08689, 2019.
- Niklas Gebauer, Michael Gastegger, and Kristof Schutt. Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules. In Advances in Neural Information Processing Systems, pp. 7566–7578, 2019.
- Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. arXiv preprint arXiv:1704.01212, 2017.
- Will Grathwohl, Ricky TQ Chen, Jesse Bettencourt, Ilya Sutskever, and David Duvenaud. Ffjord: Free-form continuous dynamics for scalable reversible generative models. arXiv preprint arXiv:1810.01367, 2018.
- Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Scholkopf, and Alexander Smola. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723–773, 2012.
- Michael Gutmann and Aapo Hyvarinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 297–304, 2010.
- Thomas A Halgren. Merck molecular force field. i. basis, form, scope, parameterization, and performance of mmff94. Journal of computational chemistry, 17(5-6):490–519, 1996a.
- Thomas A Halgren. Merck molecular force field. v. extension of mmff94 using experimental data, additional computational data, and empirical rules. Journal of Computational Chemistry, 17(5-6): 616–641, 1996b.
- Paul CD Hawkins. Conformation generation: the state of the art. Journal of Chemical Information and Modeling, 57(8):1747–1756, 2017.
- Geoffrey E Hinton and Ruslan R Salakhutdinov. Reducing the dimensionality of data with neural networks. science, 313(5786):504–507, 2006.
- Moritz Hoffmann and Frank Noe. Generating valid euclidean distance matrices. arXiv preprint arXiv:1910.03131, 2019.
- John Ingraham, Adam J Riesselman, Chris Sander, and Debora S Marks. Learning protein structure with a differentiable simulator. In International Conference on Learning Representations, 2019.
- Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
- Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and F Huang. A tutorial on energy-based learning. Predicting structured data, 1(0), 2006.
- Tobias Lemke and Christine Peter. Encodermap: Dimensionality reduction and generation of molecule conformations. Journal of chemical theory and computation, 15(2):1209–1215, 2019.
- Leo Liberti, Carlile Lavor, Nelson Maculan, and Antonio Mucherino. Euclidean distance geometry and applications. SIAM review, 56(1):3–69, 2014.
- Elman Mansimov, Omar Mahmood, Seokho Kang, and Kyunghyun Cho. Molecular geometry prediction using a deep generative graph neural network. arXiv preprint arXiv:1904.00314, 2019.
- Jiquan Ngiam, Zhenghao Chen, Pang W Koh, and Andrew Y Ng. Learning deep energy models. In Proceedings of the 28th international conference on machine learning (ICML-11), pp. 1105– 1112, 2011.
- Frank Noe, Simon Olsson, Jonas Kohler, and Hao Wu. Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning. Science, 365(6457):eaaw1147, 2019.
- Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
- Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1(1):1–7, 2014.
- Anthony K Rappe, Carla J Casewit, KS Colwell, William A Goddard III, and W Mason Skiff. Uff, a full periodic table force field for molecular mechanics and molecular dynamics simulations. Journal of the American chemical society, 114(25):10024–10035, 1992.
- Danilo Jimenez Rezende and Shakir Mohamed. Variational inference with normalizing flows. arXiv preprint arXiv:1505.05770, 2015.
- Sereina Riniker and Gregory A Landrum. Better informed distance geometry: using what we know to improve conformation generation. Journal of chemical information and modeling, 55(12): 2562–2574, 2015.
- Kristof Schutt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert Muller. Schnet: A continuous-filter convolutional neural network for modeling quantum interactions. In Advances in neural information processing systems, pp. 991–1001, 2017.
- Andrew W Senior, Richard Evans, John Jumper, James Kirkpatrick, Laurent Sifre, Tim Green, Chongli Qin, Augustin Zıdek, Alexander WR Nelson, Alex Bridgland, et al. Improved protein structure prediction using potentials from deep learning. Nature, 577(7792):706–710, 2020.
- Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: a flow-based autoregressive model for molecular graph generation. arXiv preprint arXiv:2001.09382, 2020.
- Gregor NC Simm and Jose Miguel Hernandez-Lobato. A generative model for molecular distance geometry. In International Conference on Machine Learning, 2020.
- Justin S Smith, Olexandr Isayev, and Adrian E Roitberg. Ani-1: an extensible neural network potential with dft accuracy at force field computational cost. Chemical science, 8(4):3192–3203, 2017.
- Jianwen Xie, Yang Lu, Song-Chun Zhu, and Yingnian Wu. A theory of generative convnet. In International Conference on Machine Learning, pp. 2635–2644, 2016.
- Jianwen Xie, Song-Chun Zhu, and Ying Nian Wu. Synthesizing dynamic patterns by spatialtemporal generative convnet. In Proceedings of the ieee conference on computer vision and pattern recognition, pp. 7093–7101, 2017.
- Jianwen Xie, Yang Lu, Ruiqi Gao, and Ying Nian Wu. Cooperative learning of energy-based model and latent variable model via mcmc teaching. In AAAI, volume 1, pp. 7, 2018a.
- Jianwen Xie, Yang Lu, Ruiqi Gao, Song-Chun Zhu, and Ying Nian Wu. Cooperative training of descriptor and generator networks. IEEE transactions on pattern analysis and machine intelligence, 42(1):27–45, 2018b.
- Jianwen Xie, Zilong Zheng, Ruiqi Gao, Wenguan Wang, Song-Chun Zhu, and Ying Nian Wu. Learning descriptor networks for 3d shape synthesis and analysis. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8629–8638, 2018c.
- Jianwen Xie, Song-Chun Zhu, and Ying Nian Wu. Learning energy-based spatial-temporal generative convnets for dynamic patterns. IEEE transactions on pattern analysis and machine intelligence, 2019.
- Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. In Advances in neural information processing systems, pp. 6410–6421, 2018.
- Inspired by classic molecular distance geometry (Crippen et al., 1988), we also generate the confirmations by firstly predicting all the pairwise distances, which enjoys the invariant property to rotation and translation. Since the bonds existing in the molecular graph are not sufficient to characterize a conformation, we pre-process the graphs by extending auxiliary edges. Specifically, the atoms that are 2 or 3 hops away are connected with virtual bonds, labeled differently from the real bonds of the original graph. These extra edges contribute to reducing the degrees of freedom in the 3D coordinates, with the edges between second neighbors helping to fix the angles between atoms, and those between third neighbors fixing dihedral angles.
- Our model is implemented in PyTorch (Paszke et al., 2017). The MPNN in CGCF is implemented with 3 layers, and the embedding dimension is set as 128. And the SchNet in ETM is implemented with 6 layers with the embedding dimension set as 128. We train our CGCF with a batch size of 128 and a learning rate of 0.001 until convergence. After obtaining the CGCF, we train the ETM with a batch size of 384 and a learning rate of 0.001 until convergence. For all experimental settings, we use Adam (Kingma & Ba, 2014) to optimize our model.

Tags

Comments

数据免责声明

页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果，我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问，可以通过电子邮件方式联系我们：report@aminer.cn