A Q-learning method based on coarse-to-fine potential energy surface for locating transition state and reaction pathway

Transition state (TS) on the potential energy surface (PES) plays a key role in determining the kinetics and thermodynamics of chemical reactions. Inspired by the fact that the dynamics of complex systems are always driven by rare but significant transition events, we herein propose a TS search method in accordance with the Q-learning algorithm. Appropriate reward functions are set for a given PES to optimize the reaction pathway through continuous trial and error, and then the TS can be obtained from the optimized reaction pathway. The validity of this Q-learning method with reasonable settings of Q-value table including actions, states, learning rate, greedy rate, discount rate, and so on, is exemplified in 2 two-dimensional potential functions. In the applications of the Q-learning method to two chemical reactions, it is demonstrated that the Q-learning method can predict consistent TS and reaction pathway with those by ab initio calculations. Notably, the PES must be well prepared before using the Q-learning method, and a coarse-to-fine PES scanning scheme is thus introduced to save the computational time while maintaining the accuracy of the Q-learning prediction. This work offers a simple and reliable Q-learning method to search for all possible TS and reaction pathway of a chemical reaction, which may be a new option for effectively exploring the PES in an extensive search manner.