Soft Committee Machine Using Simple Derivative Term
ARTIFICIAL INTELLIGENCE AND SOFT COMPUTING ICAISC 2014, PT I(2014)
摘要
In on-line gradient descent learning, the local property of the derivative of the output function can cause slow convergence. This phenomenon, called a plateau, occurs in the learning process of the multilayer network. Improving the derivative term, we employ the proposed method replacing the derivative term with a constant that greatly increases the relaxation speed. Moreover, we replace the derivative term with the 2nd order of expansion of the derivative, and it beaks a plateau faster than the original method.
更多查看译文
关键词
soft committee machine,derivative,Taylor expansion,relaxation speed,residual error,statistical mechanics
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络