Width Of Minima Reached By Stochastic Gradient Descent Is Influenced By Learning Rate To Batch Size Ratio

We show that the dynamics and convergence properties of SGD are set by the ratio of learning rate to batch size. We observe that this ratio is a key determinant of the generalization error, which we suggest is mediated by controlling the width of the final minima found by SGD. We verify our analysis experimentally on a range of deep neural networks and datasets.