AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
We introduced Graph Convolutional Policy Network, a graph generation policy network using graph state representation and adversarial training, and applied it to the task of goal-directed molecular graph generation

Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation.

ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 31 (NIPS 2018), (2019): 6410-6421

Cited by: 291|Views360
EI
Full Text
Bibtex
Weibo

Abstract

Generating novel graph structures that optimize given objectives while obeying some given underlying rules is fundamental for chemistry, biology and social science research. This is especially important in the task of molecular graph generation, whose goal is to discover novel molecules with desired properties such as drug-likeness and sy...More

Code:

Data:

0
Introduction
  • Many important problems in drug discovery and material science are based on the principle of designing molecular structures with specific desired properties.
  • This remains a challenging task due to the large size of chemical space.
  • The generation of novel and valid molecular graphs that can directly optimize various desired physical, chemical and biological property objectives remains to be a challenging task, since these property objectives are highly complex [37] and non-differentiable.
  • The generation model should be able to actively explore the vast chemical space, as the distribution of the molecules that possess those desired properties does not necessarily match the distribution of molecules from existing datasets
Highlights
  • Many important problems in drug discovery and material science are based on the principle of designing molecular structures with specific desired properties
  • We propose Graph Convolutional Policy Network (GCPN), an approach to generate molecules where the generation process can be guided towards specified desired objectives, while restricting the output space based on underlying chemical rules
  • We evaluate Graph Convolutional Policy Network in three distinct molecule generation tasks that are relevant to drug discovery and materials science: molecule property optimization, property targeting and conditional property optimization
  • We describe the problem definition, the environment design, and the Graph Convolutional Policy Network that predicts a distribution of actions which are used to update the graph being generated
  • We use a 3-layer defined Graph Convolutional Policy Network as the policy network with 64 dimensional node embedding in all hidden layers, and batch normalization [13] is applied after each layer
  • Graph Convolutional Policy Network generates molecules with property scores 61% higher than the best baseline method, and outperforms the baseline models in the constrained optimization setting by 184% on average
  • We introduced Graph Convolutional Policy Network, a graph generation policy network using graph state representation and adversarial training, and applied it to the task of goal-directed molecular graph generation
Methods
  • The authors formulate the problem of graph generation as learning an RL agent that iteratively adds substructures and edges to the molecular graph in a chemistry-aware environment.
  • The authors use a 3-layer defined GCPN as the policy network with 64 dimensional node embedding in all hidden layers, and batch normalization [13] is applied after each layer
  • Another 3-layer GCN with the same architecture is used for discriminator training.
  • The authors observe comparable performance among different aggregation functions and select SUM(·) for all experiments
  • The authors found both the expert pretraining and RL objective important for generating high quality molecules, both of them are kept throughout training.
  • Both objectives are trained using Adam optimizer [19] with batch size 32
Results
  • Penalized logP is a logP score that accounts for ring size and synthetic accessibility [6], while QED is an indicator of drug-likeness
  • Note that both scores are calculated from empirical prediction models whose parameters are estimated from related datasets [41, 1], and these scores are widely used in previous molecule generation papers [9, 22, 4, 39, 27].
  • The authors adopt the same evaluation method in previous approaches [22, 4, 16], reporting the best 3 property scores found by
Conclusion
  • The authors introduced GCPN, a graph generation policy network using graph state representation and adversarial training, and applied it to the task of goal-directed molecular graph generation.
  • GCPN consistently outperforms other state-of-the-art approaches in the tasks of molecular property optimization and targeting, and at the same time, maintains 100% validity and resemblance to realistic molecules.
  • The application of GCPN can extend well beyond molecule generation.
  • The algorithm can be applied to generate graphs in many contexts, such as electric circuits, social networks, and explore graphs that can optimize certain domain specific properties
Summary
  • Introduction:

    Many important problems in drug discovery and material science are based on the principle of designing molecular structures with specific desired properties.
  • This remains a challenging task due to the large size of chemical space.
  • The generation of novel and valid molecular graphs that can directly optimize various desired physical, chemical and biological property objectives remains to be a challenging task, since these property objectives are highly complex [37] and non-differentiable.
  • The generation model should be able to actively explore the vast chemical space, as the distribution of the molecules that possess those desired properties does not necessarily match the distribution of molecules from existing datasets
  • Methods:

    The authors formulate the problem of graph generation as learning an RL agent that iteratively adds substructures and edges to the molecular graph in a chemistry-aware environment.
  • The authors use a 3-layer defined GCPN as the policy network with 64 dimensional node embedding in all hidden layers, and batch normalization [13] is applied after each layer
  • Another 3-layer GCN with the same architecture is used for discriminator training.
  • The authors observe comparable performance among different aggregation functions and select SUM(·) for all experiments
  • The authors found both the expert pretraining and RL objective important for generating high quality molecules, both of them are kept throughout training.
  • Both objectives are trained using Adam optimizer [19] with batch size 32
  • Results:

    Penalized logP is a logP score that accounts for ring size and synthetic accessibility [6], while QED is an indicator of drug-likeness
  • Note that both scores are calculated from empirical prediction models whose parameters are estimated from related datasets [41, 1], and these scores are widely used in previous molecule generation papers [9, 22, 4, 39, 27].
  • The authors adopt the same evaluation method in previous approaches [22, 4, 16], reporting the best 3 property scores found by
  • Conclusion:

    The authors introduced GCPN, a graph generation policy network using graph state representation and adversarial training, and applied it to the task of goal-directed molecular graph generation.
  • GCPN consistently outperforms other state-of-the-art approaches in the tasks of molecular property optimization and targeting, and at the same time, maintains 100% validity and resemblance to realistic molecules.
  • The application of GCPN can extend well beyond molecule generation.
  • The algorithm can be applied to generate graphs in many contexts, such as electric circuits, social networks, and explore graphs that can optimize certain domain specific properties
Tables
  • Table1: Comparison of the top 3 property scores of generated molecules found by each model
  • Table2: Comparison of the effectiveness of property targeting task
  • Table3: Comparison of the performance in the constrained optimization task
Download tables as Excel
Related work
  • Yang et al [42] and Olivecrona et al [31] proposed a recurrent neural network (RNN) SMILES string generator with molecular properties as objective that is optimized using Monte Carlo tree search and policy gradient respectively. Guimaraes et al [27] and Sanchez-Lengeling et al [34] further included an adversarial loss to the reinforcement learning reward to enforce similarity to a given molecule dataset. In contrast, instead of using a text-based molecular representation, our approach uses a graph-based molecular representation, which leads to many important benefits as discussed in the introduction. Jin et al [16] proposed to use a variational autoencoder (VAE) framework, where the molecules are represented as junction trees of small clusters of atoms. This approach can only indirectly optimize molecular properties in the learned latent embedding space before decoding to a molecule, whereas our approach can directly optimize molecular properties of the molecular graphs. You et al [43] used an auto-regressive model to maximize the likelihood of the graph generation process, but it cannot be used to generate attributed graphs. Li et al [25] and Li et al [26] described sequential graph generation models where conditioning labels can be incorporated to generate molecules whose molecular properties are close to specified target scores. However, these approaches are also unable to directly perform optimization on desired molecular properties. Overall, modeling the goal-directed graph generation task in a reinforcement learning framework is still largely unexplored.
Funding
  • This research has been supported in part by DARPA SIMPLEX, ARO MURI, Stanford Data Science Initiative, Huawei, JD, and Chan Zuckerberg Biohub
  • The Pande Group acknowledges the generous support of Dr
  • The Pande Group is broadly supported by grants from the NIH (R01 GM062868 and U19 AI109662) as well as gift funds and contributions from Folding@home donors
Reference
  • G. R. Bickerton, G. V. Paolini, J. Besnard, S. Muresan, and A. L. Hopkins. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90, 2012.
    Google ScholarLocate open access versionFindings
  • K. H. Bleicher, H.-J. Böhm, K. Müller, and A. I. Alanine. Hit and lead generation: beyond high-throughput screening. Nature Reviews Drug Discovery, 2:369–378, 2003.
    Google ScholarLocate open access versionFindings
  • G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba. Openai gym. CoRR, abs/1606.01540, 2016.
    Findings
  • H. Dai, Y. Tian, B. Dai, S. Skiena, and L. Song. Syntax-directed variational autoencoder for structured data. arXiv preprint arXiv:1802.08786, 2018.
    Findings
  • D. K. Duvenaud, D. Maclaurin, J. Iparraguirre, R. Bombarell, T. Hirzel, A. Aspuru-Guzik, and R. P. Adams. Convolutional networks on graphs for learning molecular fingerprints. In Advances in neural information processing systems, 2015.
    Google ScholarLocate open access versionFindings
  • P. Ertl. Estimation of synthetic accessibility score of drug-like molecules. J. Cheminform, 2009.
    Google ScholarLocate open access versionFindings
  • P. Ertl, R. Lewis, E. J. Martin, and V. Polyakov. In silico generation of novel, drug-like chemical matter using the LSTM neural network. CoRR, abs/1712.07449, 2017.
    Findings
  • J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl. Neural message passing for quantum chemistry, 2017.
    Google ScholarFindings
  • R. Gómez-Bombarelli, J. N. Wei, D. Duvenaud, J. M. Hernández-Lobato, B. Sánchez-Lengeling, D. Sheberla, J. Aguilera-Iparraguirre, T. D. Hirzel, R. P. Adams, and A. Aspuru-Guzik. Automatic chemical design using a data-driven continuous representation of molecules. ACS Central Science, 2016.
    Google ScholarLocate open access versionFindings
  • I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in neural information processing systems, 2014.
    Google ScholarLocate open access versionFindings
  • T. A. Halgren. Merck molecular force field. i. basis, form, scope, parameterization, and performance of mmff94. Journal of computational chemistry, 17(5-6):490–519, 1996.
    Google ScholarLocate open access versionFindings
  • W. Hamilton, Z. Ying, and J. Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, 2017.
    Google ScholarLocate open access versionFindings
  • S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In F. Bach and D. Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pages 448–456, Lille, France, 07–09 Jul 2015. PMLR.
    Google ScholarLocate open access versionFindings
  • J. J. Irwin, T. Sterling, M. M. Mysinger, E. S. Bolstad, and R. G. Coleman. Zinc: a free tool to discover chemistry for biology. Journal of chemical information and modeling, 52(7):1757– 1768, 2012.
    Google ScholarLocate open access versionFindings
  • E. Jannik Bjerrum and R. Threlfall. Molecular Generation with Recurrent Neural Networks (RNNs). arXiv preprint arXiv:1705.04612, 2017.
    Findings
  • W. Jin, R. Barzilay, and T. Jaakkola. Junction tree variational autoencoder for molecular graph generation. arXiv preprint arXiv:1802.04364, 2018.
    Findings
  • S. Kakade and J. Langford. Approximately optimal approximate reinforcement learning. In International Conference on Machine Learning, 2002.
    Google ScholarLocate open access versionFindings
  • S. Kearnes, K. McCloskey, M. Berndl, V. Pande, and P. Riley. Molecular graph convolutions: moving beyond fingerprints. Journal of Computer-Aided Molecular Design, 30:595–608, Aug. 2016.
    Google ScholarLocate open access versionFindings
  • D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. International Conference on Learning Representations, 2015.
    Google ScholarLocate open access versionFindings
  • T. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2016.
    Google ScholarLocate open access versionFindings
  • P. Kirkpatrick and C. Ellis. Chemical space. Nature, 432:823 EP –, Dec 2004.
    Google ScholarLocate open access versionFindings
  • M. J. Kusner, B. Paige, and J. M. Hernández-Lobato. Grammar variational autoencoder. In D. Precup and Y. W. Teh, editors, International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, International Convention Centre, Sydney, Australia, 06–11 Aug 2017. PMLR.
    Google ScholarLocate open access versionFindings
  • G. Landrum. Rdkit: Open-source cheminformatics. 2006. Google Scholar, 2006.
    Google ScholarLocate open access versionFindings
  • S. Levine and V. Koltun. Guided policy search. In International Conference on Machine Learning, 2013.
    Google ScholarLocate open access versionFindings
  • Y. Li, O. Vinyals, C. Dyer, R. Pascanu, and P. Battaglia. Learning deep generative models of graphs. arXiv preprint arXiv:1803.03324, 2018.
    Findings
  • Y. Li, L. Zhang, and Z. Liu. Multi-Objective De Novo Drug Design with Conditional Graph Generative Model. ArXiv e-prints, Jan. 2018.
    Google ScholarFindings
  • G. Lima Guimaraes, B. Sanchez-Lengeling, C. Outeiral, P. L. Cunha Farias, and A. AspuruGuzik. Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models. ArXiv e-prints, May 2017.
    Google ScholarFindings
  • J. H. Lin and A. Y. H. Lu. Role of pharmacokinetics and metabolism in drug discovery and development. Pharmacological Reviews, 49(4):403–449, 1997.
    Google ScholarLocate open access versionFindings
  • C. A. Lipinski, F. Lombardo, B. W. Dominy, and P. J. Feeney. Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings. Advanced Drug Delivery Reviews, 23(1):3–25, 1997.
    Google ScholarLocate open access versionFindings
  • B. Liu, B. Ramsundar, P. Kawthekar, J. Shi, J. Gomes, Q. Luu Nguyen, S. Ho, J. Sloane, P. Wender, and V. Pande. Retrosynthetic reaction prediction using neural sequence-to-sequence models. ACS Central Science, 3(10):1103–1113, 2017.
    Google ScholarLocate open access versionFindings
  • M. Olivecrona, T. Blaschke, O. Engkvist, and H. Chen. Molecular de-novo design through deep reinforcement learning. Journal of Cheminformatics, 9(1):48, Sep 2017.
    Google ScholarLocate open access versionFindings
  • P. G. Polishchuk, T. I. Madzhidov, and A. Varnek. Estimation of the size of drug-like chemical space based on gdb-17 data. Journal of Computer-Aided Molecular Design, 27(8):675–679, Aug 2013.
    Google ScholarLocate open access versionFindings
  • D. Rogers and M. Hahn. Extended-connectivity fingerprints. Journal of chemical information and modeling, 50(5):742–754, 2010.
    Google ScholarLocate open access versionFindings
  • B. Sanchez-Lengeling, C. Outeiral, G. L. Guimaraes, and A. Aspuru-Guzik. Optimizing distributions over molecular space. An Objective-Reinforced Generative Adversarial Network for Inverse-design Chemistry (ORGANIC). ChemRxiv e-prints, 8 2017.
    Google ScholarLocate open access versionFindings
  • J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. CoRR, abs/1707.06347, 2017.
    Findings
  • K. T. Schütt, F. Arbabzadah, S. Chmiela, K. R. Müller, and A. Tkatchenko. Quantum-chemical insights from deep tensor neural networks. Nature Communications, 8:13890, Jan 2017. Article.
    Google ScholarLocate open access versionFindings
  • M. D. Segall. Multi-parameter optimization: Identifying high quality compounds with a balance of properties. Current Pharmaceutical Design, 18(9):1292–1310, 2012.
    Google ScholarLocate open access versionFindings
  • M. H. S. Segler, T. Kogej, C. Tyrchan, and M. P. Waller. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS Central Science, 4(1):120–131, 2018.
    Google ScholarLocate open access versionFindings
  • M. Simonovsky and N. Komodakis. Graphvae: Towards generation of small graphs using variational autoencoders. arXiv preprint arXiv:1802.03480, 2018.
    Findings
  • D. Weininger. Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules. Journal of chemical information and computer sciences, 28(1):31–36, 1988.
    Google ScholarLocate open access versionFindings
  • S. A. Wildman and G. M. Crippen. Prediction of physicochemical parameters by atomic contributions. Journal of Chemical Information and Computer Sciences, 39(5):868–873, 1999.
    Google ScholarLocate open access versionFindings
  • X. Yang, J. Zhang, K. Yoshizoe, K. Terayama, and K. Tsuda. ChemTS: An Efficient Python Library for de novo Molecular Generation. ArXiv e-prints, Sept. 2017.
    Google ScholarFindings
  • J. You, R. Ying, X. Ren, W. L. Hamilton, and J. Leskovec. Graphrnn: A deep generative model for graphs. arXiv preprint arXiv:1802.08773, 2018.
    Findings
  • L. Yu, W. Zhang, J. Wang, and Y. Yu. Seqgan: Sequence generative adversarial nets with policy gradient. In AAAI, 2017.
    Google ScholarFindings
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科