AI帮你理解科学

AI 生成解读视频

AI抽取解析论文重点内容自动生成视频


pub
生成解读视频

AI 溯源

AI解析本论文相关学术脉络


Master Reading Tree
生成 溯源树

AI 精读

AI抽取本论文的概要总结


微博一下
In we describe a fourth perspective based on the infinite limit of finite mixture models, and give detail for how the hierarchical Dirichlet process can be applied to the infinite hidden Markov model

Sharing Clusters among Related Groups: Hierarchical Dirichlet Processes

NIPS, (2005)

引用4318|浏览98
EI
下载 PDF 全文
引用
微博一下

摘要

We propose the hierarchical Dirichlet process (HDP), a nonparametric Bayesian model for clustering problems involving multiple groups of data. Each group of data is modeled with a mixture, with the number of components being open-ended and inferred automatically by the model. Further, components can be shared across groups, allowing depen...更多

代码

数据

简介
  • One of the most significant conceptual and practical tools in the Bayesian paradigm is the notion of a hierarchical model.
  • Building on the notion that a parameter is a random variable, hierarchical models have applications to a variety of forms of grouped or relational data and to general problems involving “multi-task learning” or “learning to learn.” A simple and classical example is the Gaussian means problem, in which a grand mean μ0 is drawn from some distribution, a set of K means are drawn independently from a Gaussian with mean μ0, and data are subsequently drawn independently from K Gaussian distributions with these means.
  • The authors want to discover topics that are common across multiple documents in the same corpus, as well as across multiple corpora
重点内容
  • One of the most significant conceptual and practical tools in the Bayesian paradigm is the notion of a hierarchical model
  • In [10] we show that the hierarchical Dirichlet process framework can be applied to obtain a cleaner formulation of the infinite hidden Markov model, providing effective new inference algorithms and potentially hierarchical extensions
  • We have described the hierarchical Dirichlet process, a hierarchical, nonparametric model for clustering problems involving multiple groups of data
  • hierarchical Dirichlet process mixture models are able to automatically determine the appropriate number of mixture components needed, and exhibit sharing of statistical strength across groups by having components shared across groups
  • We have described the hierarchical Dirichlet process as a distribution over distributions, using both the stick-breaking construction and the Chinese restaurant franchise
  • In [10] we describe a fourth perspective based on the infinite limit of finite mixture models, and give detail for how the hierarchical Dirichlet process can be applied to the infinite hidden Markov model
方法
  • Nematode biology abstracts.
  • The authors applied both models to a corpus of nematode biology abstracts1, evaluating the perplexity of both models on held out abstracts.
  • In order to study the nonparametric nature of the HDP, the authors used the same experimental setup for both models2, except that in LDA the authors had to vary the number of topics used between 10 and 120, while the HDP obtained posterior samples over this automatically
结论
  • The authors have described the hierarchical Dirichlet process, a hierarchical, nonparametric model for clustering problems involving multiple groups of data.
  • In [10] the authors describe a fourth perspective based on the infinite limit of finite mixture models, and give detail for how the HDP can be applied to the iHMM.
  • Direct extensions of the model include use of nonparametric priors other than the DP, building higher level hierarchies as in the NIPS experiment, as well as hierarchical extensions to the iHMM
表格
  • Table1: Topics shared between VS and the other sections. Shown are the two topics with most numbers of VS words, but also with significant numbers of words from the other section
Download tables as Excel
基金
  • Proposes the hierarchical Dirichlet process , a nonparametric Bayesian model for clustering problems involving multiple groups of data
  • Reports experimental results on three text corpora showing the effective and superior performance of the HDP over previous models
  • Presents a notion of a hierarchical Dirichlet process in which the base distribution G0 for a set of DPs is itself a draw from a DP
  • Presents analogous stick-breaking and Chinese restaurant representations for the HDP
引用论文
  • D.M. Blei, A.Y. Ng, and M.I. Jordan. Latent Dirichlet allocation. JMLR, 3:993–1022, 2003.
    Google ScholarLocate open access versionFindings
  • M.D. Escobar and M. West. Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association, 90:577–588, 1995.
    Google ScholarLocate open access versionFindings
  • S.N. MacEachern and P. Muller. Estimating mixture of Dirichlet process models. Journal of Computational and Graphical Statistics, 7:223–238, 1998.
    Google ScholarLocate open access versionFindings
  • T.S. Ferguson. A Bayesian analysis of some nonparametric problems. Annals of Statistics, 1(2):209–230, 1973. Springer, Berlin, 1985.
    Google ScholarLocate open access versionFindings
  • [6] J. Sethuraman. A constructive definition of Dirichlet priors. Statistica Sinica, 4:639–650, 1994.
    Google ScholarLocate open access versionFindings
  • [7] R.M. Neal. Markov chain sampling methods for Dirichlet process mixture models. Journal of Computational and Graphical Statistics, 9:249–265, 2000.
    Google ScholarLocate open access versionFindings
  • [8] C.E. Rasmussen. The infinite Gaussian mixture model. In NIPS, volume 12, 2000.
    Google ScholarLocate open access versionFindings
  • [9] D.M. Blei, T.L. Griffiths, M.I. Jordan, and J.B. Tenenbaum. Hierarchical topic models and the nested Chinese restaurant process. NIPS, 2004.
    Google ScholarLocate open access versionFindings
  • [10] Y.W. Teh, M.I. Jordan, M.J. Beal, and D.M. Blei. Hierarchical dirichlet processes. Technical Report 653, Department of Statistics, University of California at Berkeley, 2004.
    Google ScholarFindings
  • [11] M.J. Beal, Z. Ghahramani, and C.E. Rasmussen. The infinite hidden Markov model. In NIPS, volume 14, 2002.
    Google ScholarLocate open access versionFindings
  • [12] M.J. Beal. Variational Algorithms for Approximate Bayesian Inference. PhD thesis, Gatsby Unit, University College London, 2004.
    Google ScholarFindings
0
您的评分 :

暂无评分

标签
评论
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn