AI帮你理解科学

AI 生成解读视频

AI抽取解析论文重点内容自动生成视频

生成解读视频

AI 溯源

AI解析本论文相关学术脉络

生成溯源树

AI 精读

AI抽取本论文的概要总结

微博一下：

In we describe a fourth perspective based on the infinite limit of finite mixture models, and give detail for how the hierarchical Dirichlet process can be applied to the infinite hidden Markov model

Sharing Clusters among Related Groups: Hierarchical Dirichlet Processes

NIPS, (2005)

引用：4318|浏览98

下载 PDF 全文

其它链接

dblp.uni-trier.de academic.microsoft.com

引用

微博一下

关键词

hierarchical dirichlet processbayesian model

摘要

We propose the hierarchical Dirichlet process (HDP), a nonparametric Bayesian model for clustering problems involving multiple groups of data. Each group of data is modeled with a mixture, with the number of components being open-ended and inferred automatically by the model. Further, components can be shared across groups, allowing depen...更多

代码：

数据：

精读

下载 PDF 全文

PPT (上传PPT)

相似论文

引用论文

被引论文

简介

One of the most significant conceptual and practical tools in the Bayesian paradigm is the notion of a hierarchical model.
Building on the notion that a parameter is a random variable, hierarchical models have applications to a variety of forms of grouped or relational data and to general problems involving “multi-task learning” or “learning to learn.” A simple and classical example is the Gaussian means problem, in which a grand mean μ0 is drawn from some distribution, a set of K means are drawn independently from a Gaussian with mean μ0, and data are subsequently drawn independently from K Gaussian distributions with these means.
The authors want to discover topics that are common across multiple documents in the same corpus, as well as across multiple corpora

重点内容

One of the most significant conceptual and practical tools in the Bayesian paradigm is the notion of a hierarchical model
In [10] we show that the hierarchical Dirichlet process framework can be applied to obtain a cleaner formulation of the infinite hidden Markov model, providing effective new inference algorithms and potentially hierarchical extensions
We have described the hierarchical Dirichlet process, a hierarchical, nonparametric model for clustering problems involving multiple groups of data
hierarchical Dirichlet process mixture models are able to automatically determine the appropriate number of mixture components needed, and exhibit sharing of statistical strength across groups by having components shared across groups
We have described the hierarchical Dirichlet process as a distribution over distributions, using both the stick-breaking construction and the Chinese restaurant franchise
In [10] we describe a fourth perspective based on the infinite limit of finite mixture models, and give detail for how the hierarchical Dirichlet process can be applied to the infinite hidden Markov model

方法

Nematode biology abstracts.
The authors applied both models to a corpus of nematode biology abstracts1, evaluating the perplexity of both models on held out abstracts.
In order to study the nonparametric nature of the HDP, the authors used the same experimental setup for both models2, except that in LDA the authors had to vary the number of topics used between 10 and 120, while the HDP obtained posterior samples over this automatically

结论

The authors have described the hierarchical Dirichlet process, a hierarchical, nonparametric model for clustering problems involving multiple groups of data.
In [10] the authors describe a fourth perspective based on the infinite limit of finite mixture models, and give detail for how the HDP can be applied to the iHMM.
Direct extensions of the model include use of nonparametric priors other than the DP, building higher level hierarchies as in the NIPS experiment, as well as hierarchical extensions to the iHMM

表格

Table1: Topics shared between VS and the other sections. Shown are the two topics with most numbers of VS words, but also with significant numbers of words from the other section

Download tables as Excel

基金

Proposes the hierarchical Dirichlet process , a nonparametric Bayesian model for clustering problems involving multiple groups of data
Reports experimental results on three text corpora showing the effective and superior performance of the HDP over previous models
Presents a notion of a hierarchical Dirichlet process in which the base distribution G0 for a set of DPs is itself a draw from a DP
Presents analogous stick-breaking and Chinese restaurant representations for the HDP

引用论文

D.M. Blei, A.Y. Ng, and M.I. Jordan. Latent Dirichlet allocation. JMLR, 3:993–1022, 2003.
M.D. Escobar and M. West. Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association, 90:577–588, 1995.
S.N. MacEachern and P. Muller. Estimating mixture of Dirichlet process models. Journal of Computational and Graphical Statistics, 7:223–238, 1998.
T.S. Ferguson. A Bayesian analysis of some nonparametric problems. Annals of Statistics, 1(2):209–230, 1973. Springer, Berlin, 1985.
[6] J. Sethuraman. A constructive definition of Dirichlet priors. Statistica Sinica, 4:639–650, 1994.
[7] R.M. Neal. Markov chain sampling methods for Dirichlet process mixture models. Journal of Computational and Graphical Statistics, 9:249–265, 2000.
[8] C.E. Rasmussen. The infinite Gaussian mixture model. In NIPS, volume 12, 2000.
[9] D.M. Blei, T.L. Griffiths, M.I. Jordan, and J.B. Tenenbaum. Hierarchical topic models and the nested Chinese restaurant process. NIPS, 2004.
[10] Y.W. Teh, M.I. Jordan, M.J. Beal, and D.M. Blei. Hierarchical dirichlet processes. Technical Report 653, Department of Statistics, University of California at Berkeley, 2004.
[11] M.J. Beal, Z. Ghahramani, and C.E. Rasmussen. The infinite hidden Markov model. In NIPS, volume 14, 2002.
[12] M.J. Beal. Variational Algorithms for Approximate Bayesian Inference. PhD thesis, Gatsby Unit, University College London, 2004.