Hybrid Quantum-Classical Scheduling for Accelerating Neural Network Training with Newton's Gradient Descent
arxiv(2024)
摘要
Optimization techniques in deep learning are predominantly led by first-order
gradient methodologies, such as SGD. However, neural network training can
greatly benefit from the rapid convergence characteristics of second-order
optimization. Newton's GD stands out in this category, by rescaling the
gradient using the inverse Hessian. Nevertheless, one of its major bottlenecks
is matrix inversion, which is notably time-consuming in O(N^3) time with weak
scalability.
Matrix inversion can be translated into solving a series of linear equations.
Given that quantum linear solver algorithms (QLSAs), leveraging the principles
of quantum superposition and entanglement, can operate within a
polylog(N) time frame, they present a promising approach with
exponential acceleration. Specifically, one of the most recent QLSAs
demonstrates a complexity scaling of O(d·κlog(N·κ/ϵ)), depending on: size N, condition
number κ, error tolerance ϵ, quantum oracle sparsity d of
the matrix. However, this also implies that their potential exponential
advantage may be hindered by certain properties (i.e. κ and d).
We propose Q-Newton, a hybrid quantum-classical scheduler for accelerating
neural network training with Newton's GD. Q-Newton utilizes a streamlined
scheduling module that coordinates between quantum and classical linear
solvers, by estimating reducing κ and constructing d for the quantum
solver.
Our evaluation showcases the potential for Q-Newton to significantly reduce
the total training time compared to commonly used optimizers like SGD. We
hypothesize a future scenario where the gate time of quantum machines is
reduced, possibly realized by attoseconds physics. Our evaluation establishes
an ambitious and promising target for the evolution of quantum computing.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要