RNN Training along Locally Optimal Trajectoriesvia Frank-Wolfe Algorithm
ICPR, pp. 10532-10539, 2020.
We propose a novel and efficient training method for RNNs by iteratively seeking a local minima on the loss surface within a small region, and leverage this directional vector for the update, in an outer-loop. We propose to utilize the Frank-Wolfe (FW) algorithm in this context. Although, FW implicitly involves normalized gradients, whi...More
PPT (Upload PPT)