Read full paper at:
http://www.scirp.org/journal/PaperInformation.aspx?PaperID=52713#.VKH8NcCAM4
http://www.scirp.org/journal/PaperInformation.aspx?PaperID=52713#.VKH8NcCAM4
Author(s)
The logistic normal distribution has recently been
adapted via the transformation of multivariate Gaussian variables to
model the topical distribution of documents in the presence of
correlations among topics. In this paper, we propose a probit normal
alternative approach to modelling correlated topical structures. Our use
of the probit model in the context of topic discovery is novel, as many
authors have so far concentrated solely of the logistic model partly
due to the formidable inefficiency of the multinomial probit model even
in the case of very small topical spaces. We herein circumvent the
inefficiency of multinomial probit estimation by using an adaptation of
the diagonal orthant multinomial probit in the topic models context,
resulting in the ability of our topic modeling scheme to handle corpuses
with a large number of latent topics. An additional and very important
benefit of our method lies in the fact that unlike with the logistic
normal model whose non-conjugacy leads to the need for sophisticated
sampling schemes, our approach exploits the natural conjugacy inherent
in the auxiliary formulation of the probit model to achieve greater
simplicity. The application of our proposed scheme to a well-known
Associated Press corpus not only helps discover a large number of
meaningful topics but also reveals the capturing of compellingly
intuitive correlations among certain topics. Besides, our proposed
approach lends itself to even further scalability thanks to various
existing high performance algorithms and architectures capable of
handling millions of documents.
KEYWORDS
Cite this paper
Yu, X. and Fokoué, E. (2014) Probit Normal Correlated Topic Model. Open Journal of Statistics, 4, 879-888. doi: 10.4236/ojs.2014.411083.
| [1] | Blei, D.M. and Ng, A.Y., Jordan, M.I. and Lafferty, J. (2003) Latent Dirichlet Allocation. Journal of Machine Learning Research, 3. |
| [2] | Blei, D.M. and Lafferty, J.D. (2006) Correlated Topic Models. Proceedings of the 23rd International Conference on Machine Learning, MIT Press, Cambridge, Massachusetts, 113-120. |
| [3] | Mimno, D., Wallach, H.M. and Mccallum, A. (2008) Gibbs Sampling for Logistic Normal Topic Models with Graph-Based Priors. Proceedings of NIPS Workshop on Analyzing Graphs, 2008. |
| [4] | Chen, J.F., Zhu, J., Wang, Z., Zheng, X. and Zhang, B. (2013) Scalable Inference for Logistic-Normal Topic Models. In Burges, C.J.C., Bottou, L., Welling, M., Ghahramani, Z. and Weinberger, K.Q., Eds., Advances in Neural Information Processing Systems 26, Curran Associates, Inc., 2445-2453. |
| [5] | Johndrow, J., Lum, K. and Dunson, D.B. (2013) Diagonal Orthant Multinomial Probit Models. JMLR Proceedings, Volume 31 of AISTATS, 29-38. |
| [6] |
Albert, J.H. and Chib, S. (1993)
Bayesian Analysis of Binary and Polychotomous Response Data. Journal of
the American Statistical Association, 88, 669-679. http://dx.doi.org/10.1080/01621459.1993.10476321 |
| [7] | Grun, B. and Hornik, K. (2011) Topicmodels: An R Package for Fitting Topic Models. Journal of Statistical Software, 40, 1-30. |
| [8] | Salomatin, K., Yang, Y.M. and Lad, A. (2009) Multi-Field Correlated Topic Modeling. Proceedings of the SIAM International Conference on Data Mining, SDM 2009, April 30-May 2, 2009, Sparks, 628-637. |
| [9] | Yao, L.M., Mimno, D. and McCallum, A. (2009) Efficient Methods for Topic Model Inference on Streaming Document Collections. KDD 2009: Proceedings of 15th ACM SIGKDD int’l Conference on Knowledge Discovery and Data Mining, 937-946. |
| [10] | Newman, D., Asuncion, A., Smyth, P. and Welling, M. (2009) Distributed Algorithms for Topic Models. Journal of Machine Learning Research, 10, 1801-1828. |
| [11] | Smola, A. and Narayanamurthy, S. (2010) An Architecture for Parallel Topic Models. Proc. VLDB Endow., 3, 703-710. |
| [12] | Zhu, J., Chen, N., Perkins, H. and Zhang, B. (2013) Gibbs Max-Margin Topic Models with Data Augmentation. CoRR, abs/1310.2816. eww141230lx |
评论
发表评论