Delta-STN: Efficient Bilevel Optimization for Neural Networks using Structured Response Jacobians

Juhan Bae
Juhan Bae

NIPS 2020, 2020.

Cited by: 0|Views39
EI
Weibo:
We believe that ∆-Self-Tuning Networks offer a more reliable, robust, and accurate deployment of hyperparameter optimization based on hypernetwork approach, and offer an alternative method to efficiently solve bilevel optimization problems

Abstract:

Hyperparameter optimization of neural networks can be elegantly formulated as a bilevel optimization problem. While research on bilevel optimization of neural networks has been dominated by implicit differentiation and unrolling, hypernetworks such as Self-Tuning Networks (STNs) have recently gained traction due to their ability to amor...More
0
Your rating :
0

 

Tags
Comments