Delta-STN: Efficient Bilevel Optimization for Neural Networks using Structured Response Jacobians
NIPS 2020, 2020.
We believe that ∆-Self-Tuning Networks offer a more reliable, robust, and accurate deployment of hyperparameter optimization based on hypernetwork approach, and offer an alternative method to efficiently solve bilevel optimization problems
Hyperparameter optimization of neural networks can be elegantly formulated as a bilevel optimization problem. While research on bilevel optimization of neural networks has been dominated by implicit differentiation and unrolling, hypernetworks such as Self-Tuning Networks (STNs) have recently gained traction due to their ability to amor...More
PPT (Upload PPT)