In we describe a fourth perspective based on the infinite limit of finite mixture models, and give detail for how the hierarchical Dirichlet process can be applied to the infinite hidden Markov model
Sharing Clusters among Related Groups: Hierarchical Dirichlet Processes
We propose the hierarchical Dirichlet process (HDP), a nonparametric Bayesian model for clustering problems involving multiple groups of data. Each group of data is modeled with a mixture, with the number of components being open-ended and inferred automatically by the model. Further, components can be shared across groups, allowing depen...更多
下载 PDF 全文
- One of the most significant conceptual and practical tools in the Bayesian paradigm is the notion of a hierarchical model.
- Building on the notion that a parameter is a random variable, hierarchical models have applications to a variety of forms of grouped or relational data and to general problems involving “multi-task learning” or “learning to learn.” A simple and classical example is the Gaussian means problem, in which a grand mean μ0 is drawn from some distribution, a set of K means are drawn independently from a Gaussian with mean μ0, and data are subsequently drawn independently from K Gaussian distributions with these means.
- The authors want to discover topics that are common across multiple documents in the same corpus, as well as across multiple corpora
- One of the most significant conceptual and practical tools in the Bayesian paradigm is the notion of a hierarchical model
- In  we show that the hierarchical Dirichlet process framework can be applied to obtain a cleaner formulation of the infinite hidden Markov model, providing effective new inference algorithms and potentially hierarchical extensions
- We have described the hierarchical Dirichlet process, a hierarchical, nonparametric model for clustering problems involving multiple groups of data
- hierarchical Dirichlet process mixture models are able to automatically determine the appropriate number of mixture components needed, and exhibit sharing of statistical strength across groups by having components shared across groups
- We have described the hierarchical Dirichlet process as a distribution over distributions, using both the stick-breaking construction and the Chinese restaurant franchise
- In  we describe a fourth perspective based on the infinite limit of finite mixture models, and give detail for how the hierarchical Dirichlet process can be applied to the infinite hidden Markov model
- Nematode biology abstracts.
- The authors applied both models to a corpus of nematode biology abstracts1, evaluating the perplexity of both models on held out abstracts.
- In order to study the nonparametric nature of the HDP, the authors used the same experimental setup for both models2, except that in LDA the authors had to vary the number of topics used between 10 and 120, while the HDP obtained posterior samples over this automatically
- The authors have described the hierarchical Dirichlet process, a hierarchical, nonparametric model for clustering problems involving multiple groups of data.
- In  the authors describe a fourth perspective based on the infinite limit of finite mixture models, and give detail for how the HDP can be applied to the iHMM.
- Direct extensions of the model include use of nonparametric priors other than the DP, building higher level hierarchies as in the NIPS experiment, as well as hierarchical extensions to the iHMM
- Table1: Topics shared between VS and the other sections. Shown are the two topics with most numbers of VS words, but also with significant numbers of words from the other section
- Proposes the hierarchical Dirichlet process , a nonparametric Bayesian model for clustering problems involving multiple groups of data
- Reports experimental results on three text corpora showing the effective and superior performance of the HDP over previous models
- Presents a notion of a hierarchical Dirichlet process in which the base distribution G0 for a set of DPs is itself a draw from a DP
- Presents analogous stick-breaking and Chinese restaurant representations for the HDP
- D.M. Blei, A.Y. Ng, and M.I. Jordan. Latent Dirichlet allocation. JMLR, 3:993–1022, 2003.
- M.D. Escobar and M. West. Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association, 90:577–588, 1995.
- S.N. MacEachern and P. Muller. Estimating mixture of Dirichlet process models. Journal of Computational and Graphical Statistics, 7:223–238, 1998.
- T.S. Ferguson. A Bayesian analysis of some nonparametric problems. Annals of Statistics, 1(2):209–230, 1973. Springer, Berlin, 1985.
-  J. Sethuraman. A constructive definition of Dirichlet priors. Statistica Sinica, 4:639–650, 1994.
-  R.M. Neal. Markov chain sampling methods for Dirichlet process mixture models. Journal of Computational and Graphical Statistics, 9:249–265, 2000.
-  C.E. Rasmussen. The infinite Gaussian mixture model. In NIPS, volume 12, 2000.
-  D.M. Blei, T.L. Griffiths, M.I. Jordan, and J.B. Tenenbaum. Hierarchical topic models and the nested Chinese restaurant process. NIPS, 2004.
-  Y.W. Teh, M.I. Jordan, M.J. Beal, and D.M. Blei. Hierarchical dirichlet processes. Technical Report 653, Department of Statistics, University of California at Berkeley, 2004.
-  M.J. Beal, Z. Ghahramani, and C.E. Rasmussen. The infinite hidden Markov model. In NIPS, volume 14, 2002.
-  M.J. Beal. Variational Algorithms for Approximate Bayesian Inference. PhD thesis, Gatsby Unit, University College London, 2004.