A Stochastic Machine Learning Algorithm Based on a Cautious BFGS Update Formula

With the rapid development of data-intensive applications, including large models for spine analysis, efficient stochastic optimization methods have become increasingly important. A new stochastic quasi-Newton method (SQN) is proposed for solving stochastic optimization problems. Unlike classical stochastic BFGS-type methods, an adaptive scaling damped BFGS strategy is introduced, thereby improving the stability and reliability of curvature information in stochastic settings. In addition, a mini-batch technique is incorporated into the algorithm to reduce the computational cost while preserving useful stochastic gradient information. The global convergence of the proposed method is established under diminishing step sizes and Armijo line search. A notable feature of the theoretical analysis is that the boundedness of H_k, which is commonly imposed as an additional assumption in classical methods, can be obtained automatically during the proof process. Moreover, the boundedness analysis of the approximate Hessian matrix B_k is significantly simplified through a concise and transparent proof framework. Numerical experiments are done, and the results show the superiority of proposed algorithm.