Fast and Provable Algorithms for Learning Two-Layer Polynomial Neural Networks
IEEE Transactions on Signal Processing(2019)
摘要
In this paper, we bridge the problem of (provably) learning shallow neural networks with the well-studied problem of low-rank matrix estimation. In particular, we consider two-layer networks with quadratic activations, and focus on the under-parameterized regime where the number of neurons in the hidden layer is smaller than the dimension of the input. Our main approach is to “lift” the learning problem into a higher dimension, which enables us to borrow algorithmic techniques from low-rank matrix estimation. Using this intuition, we propose three novel, non-convex training algorithms. We support our algorithms with rigorous theoretical analysis, and show that all three enjoy a linear convergence, fast running time per iteration, and near-optimal sample complexity. Finally, we complement our theoretical results with numerical experiments.
更多查看译文
关键词
Signal processing algorithms,Estimation,Biological neural networks,Approximation algorithms,Complexity theory,Symmetric matrices,Convergence
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络