Nonparametric Classification on Low Dimensional Manifolds using Overparameterized Convolutional Residual Networks
CoRR(2023)
摘要
Convolutional residual neural networks (ConvResNets), though
overparameterized, can achieve remarkable prediction performance in practice,
which cannot be well explained by conventional wisdom. To bridge this gap, we
study the performance of ConvResNeXts, which cover ConvResNets as a special
case, trained with weight decay from the perspective of nonparametric
classification. Our analysis allows for infinitely many building blocks in
ConvResNeXts, and shows that weight decay implicitly enforces sparsity on these
blocks. Specifically, we consider a smooth target function supported on a
low-dimensional manifold, then prove that ConvResNeXts can adapt to the
function smoothness and low-dimensional structures and efficiently learn the
function without suffering from the curse of dimensionality. Our findings
partially justify the advantage of overparameterized ConvResNeXts over
conventional machine learning models.
更多查看译文
关键词
low dimensional manifolds,overparameterized convolutional residual networks,nonparametric,classification
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要