AutoLR: an evolutionary approach to learning rate policies
Genetic and Evolutionary Computation Conference(2020)
摘要
ABSTRACTThe choice of a proper learning rate is paramount for good Artificial Neural Network training and performance. In the past, one had to rely on experience and trial-and-error to find an adequate learning rate. Presently, a plethora of state of the art automatic methods exist that make the search for a good learning rate easier. While these techniques are effective and have yielded good results over the years, they are general solutions. This means the optimization of learning rate for specific network topologies remains largely unexplored. This work presents AutoLR, a framework that evolves Learning Rate Schedulers for a specific Neural Network Architecture using Structured Grammatical Evolution. The system was used to evolve learning rate policies that were compared with a commonly used baseline value for learning rate. Results show that training performed using certain evolved policies is more efficient than the established baseline and suggest that this approach is a viable means of improving a neural network's performance.
更多查看译文
关键词
Learning Rate Schedulers, Structured Grammatical Evolution
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络