Gradient Estimation with Stochastic Softmax Tricks

Paulus Max B.

[0]

[0]

[0]

[0]

[0]

NIPS 2020, 2020.

被引用：1|引用|浏览52

其它链接：arxiv.org|dblp.uni-trier.de|academic.microsoft.com

关键词：

mean squared errorrelaxed gradientGumbel-Softmax trickGumbel-Max trickNeural Relational Inference更多(12+)

微博一下：

We introduced stochastic softmax tricks, which are random convex programs that capture a large class of relaxed distributions over structured, combinatorial spaces

摘要：

The Gumbel-Max trick is the basis of many relaxed gradient estimators. These estimators are easy to implement and low variance, but the goal of scaling them comprehensively to large combinatorial distributions is still outstanding. Working within the perturbation model framework, we introduce stochastic softmax tricks, which generalize ...更多

代码：

数据：

精读

下载 PDF 全文

PPT (上传PPT)

简介

Gradient computation is the methodological backbone of deep learning, but computing gradients is not always easy.
The Gumbel-Softmax estimator is the simplest; it continuously approximates the GumbelMax trick to admit a reparameterization gradient [37, 67, 71].
This is used to optimize the “soft” approximation of the loss as a surrogate for the “hard” discrete objective.

重点内容

Gradient computation is the methodological backbone of deep learning, but computing gradients is not always easy
We address gradient estimation for discrete distributions with an emphasis on latent variable models
We introduce stochastic softmax tricks (SSTs), which are a unified framework for designing structured relaxations of combinatorial distributions
Relaxed gradient estimators assume that L is differentiable and use a change of variables to remove the dependence of pθ on θ, known as the reparameterization trick [37, 67]
The Gumbel-Softmax trick (GST) [52, 35] is a simple relaxed gradient estimator for one-hot embeddings, which is based on the Gumbel-Max trick (GMT) [51, 53]
We introduced stochastic softmax tricks, which are random convex programs that capture a large class of relaxed distributions over structured, combinatorial spaces

方法

The authors' goal in these experiments was to evaluate the use of SSTs for learning distributions over structured latent spaces in deep structured models.
For NRI, the authors implemented the standard single-loss-evaluation score function estimators (REINFORCE [82] and NVIL [59]), but struggled to achieve competitive results, see App. C.
All SST models were trained with the “soft” SST and evaluated with the “hard” SMT.
The authors selected models on a validation set according to the best objective value obtained during training.

结论

The authors introduced stochastic softmax tricks, which are random convex programs that capture a large class of relaxed distributions over structured, combinatorial spaces.
The authors designed stochastic softmax tricks for subset selection and a variety of spanning tree distributions.
The authors tested their use in deep latent variable models, and found that they can be used to improve performance and to encourage the unsupervised discovery of true latent structure.
Some combinatorial objects might benefit from a more careful design of the utility distribution, while others, e.g., matchings, are still waiting to have their tricks designed

总结

Introduction:
Gradient computation is the methodological backbone of deep learning, but computing gradients is not always easy.
The Gumbel-Softmax estimator is the simplest; it continuously approximates the GumbelMax trick to admit a reparameterization gradient [37, 67, 71].
This is used to optimize the “soft” approximation of the loss as a surrogate for the “hard” discrete objective.
Methods:
The authors' goal in these experiments was to evaluate the use of SSTs for learning distributions over structured latent spaces in deep structured models.
For NRI, the authors implemented the standard single-loss-evaluation score function estimators (REINFORCE [82] and NVIL [59]), but struggled to achieve competitive results, see App. C.
All SST models were trained with the “soft” SST and evaluated with the “hard” SMT.
The authors selected models on a validation set according to the best objective value obtained during training.
Conclusion:
The authors introduced stochastic softmax tricks, which are random convex programs that capture a large class of relaxed distributions over structured, combinatorial spaces.
The authors designed stochastic softmax tricks for subset selection and a variety of spanning tree distributions.
The authors tested their use in deep latent variable models, and found that they can be used to improve performance and to encourage the unsupervised discovery of true latent structure.
Some combinatorial objects might benefit from a more careful design of the utility distribution, while others, e.g., matchings, are still waiting to have their tricks designed

表格

Table1: Table 1
Table2: Matching ground truth structure (non-tree → tree) improves performance on ListOps. The utility distribution impacts performance. Test task accuracy and structure recovery metrics are shown from models selected on valid. task accuracy
Table3: Table 3
Table4: For k-subset selection on appearance aspect, SSTs select subsets with high precision and outperform baseline relaxations. Test set MSE and subset precision is shown for models selected on valid. MSE
Table5: For k-subset selection on palate aspect, SSTs tend to outperform baseline relaxations. Test set MSE and subset precision is shown for models selected on valid. MSE
Table6: For k-subset selection on taste aspect, MSE and subset precision tend to be lower for all methods. This is because the taste rating is highly correlated with other ratings making it difficult to identify subsets with high precision. SSTs achieve small improvements. Test set MSE and subset precision is shown for models selected on valid. MSE
Table7: NVIL and REINFORCE fails to get competitive results to their SST counterparts. Top |V | − 1 and Spanning Tree fail to learn edge structure for both REINFORCE and NVIL

Download tables as Excel

Gradient Estimation with Stochastic Softmax Tricks

Introduction:

Methods:

Conclusion: