跳至主要内容

Probit Normal Correlated Topic Model

Read full paper at:
http://www.scirp.org/journal/PaperInformation.aspx?PaperID=52713#.VKH8NcCAM4

Author(s)   
The logistic normal distribution has recently been adapted via the transformation of multivariate Gaussian variables to model the topical distribution of documents in the presence of correlations among topics. In this paper, we propose a probit normal alternative approach to modelling correlated topical structures. Our use of the probit model in the context of topic discovery is novel, as many authors have so far concentrated solely of the logistic model partly due to the formidable inefficiency of the multinomial probit model even in the case of very small topical spaces. We herein circumvent the inefficiency of multinomial probit estimation by using an adaptation of the diagonal orthant multinomial probit in the topic models context, resulting in the ability of our topic modeling scheme to handle corpuses with a large number of latent topics. An additional and very important benefit of our method lies in the fact that unlike with the logistic normal model whose non-conjugacy leads to the need for sophisticated sampling schemes, our approach exploits the natural conjugacy inherent in the auxiliary formulation of the probit model to achieve greater simplicity. The application of our proposed scheme to a well-known Associated Press corpus not only helps discover a large number of meaningful topics but also reveals the capturing of compellingly intuitive correlations among certain topics. Besides, our proposed approach lends itself to even further scalability thanks to various existing high performance algorithms and architectures capable of handling millions of documents.
Cite this paper
Yu, X. and Fokoué, E. (2014) Probit Normal Correlated Topic Model. Open Journal of Statistics, 4, 879-888. doi: 10.4236/ojs.2014.411083.
 

[1] Blei, D.M. and Ng, A.Y., Jordan, M.I. and Lafferty, J. (2003) Latent Dirichlet Allocation. Journal of Machine Learning Research, 3.
[2] Blei, D.M. and Lafferty, J.D. (2006) Correlated Topic Models. Proceedings of the 23rd International Conference on Machine Learning, MIT Press, Cambridge, Massachusetts, 113-120.
[3] Mimno, D., Wallach, H.M. and Mccallum, A. (2008) Gibbs Sampling for Logistic Normal Topic Models with Graph-Based Priors. Proceedings of NIPS Workshop on Analyzing Graphs, 2008.
[4] Chen, J.F., Zhu, J., Wang, Z., Zheng, X. and Zhang, B. (2013) Scalable Inference for Logistic-Normal Topic Models. In Burges, C.J.C., Bottou, L., Welling, M., Ghahramani, Z. and Weinberger, K.Q., Eds., Advances in Neural Information Processing Systems 26, Curran Associates, Inc., 2445-2453.
[5] Johndrow, J., Lum, K. and Dunson, D.B. (2013) Diagonal Orthant Multinomial Probit Models. JMLR Proceedings, Volume 31 of AISTATS, 29-38.
[6] Albert, J.H. and Chib, S. (1993) Bayesian Analysis of Binary and Polychotomous Response Data. Journal of the American Statistical Association, 88, 669-679.
http://dx.doi.org/10.1080/01621459.1993.10476321
[7] Grun, B. and Hornik, K. (2011) Topicmodels: An R Package for Fitting Topic Models. Journal of Statistical Software, 40, 1-30.
[8] Salomatin, K., Yang, Y.M. and Lad, A. (2009) Multi-Field Correlated Topic Modeling. Proceedings of the SIAM International Conference on Data Mining, SDM 2009, April 30-May 2, 2009, Sparks, 628-637.
[9] Yao, L.M., Mimno, D. and McCallum, A. (2009) Efficient Methods for Topic Model Inference on Streaming Document Collections. KDD 2009: Proceedings of 15th ACM SIGKDD int’l Conference on Knowledge Discovery and Data Mining, 937-946.
[10] Newman, D., Asuncion, A., Smyth, P. and Welling, M. (2009) Distributed Algorithms for Topic Models. Journal of Machine Learning Research, 10, 1801-1828.
[11] Smola, A. and Narayanamurthy, S. (2010) An Architecture for Parallel Topic Models. Proc. VLDB Endow., 3, 703-710.
[12] Zhu, J., Chen, N., Perkins, H. and Zhang, B. (2013) Gibbs Max-Margin Topic Models with Data Augmentation. CoRR, abs/1310.2816.  eww141230lx
标签：

评论

发表评论

此博客中的热门博文

Does Immigration Promote the Investment of the Monopolistic Firm?

In the present paper, we examine the effect of increasing uncertainty of immigrants’ growth on the optimal timing of investment of a firm that has a monopolistic power over the labor market. It is revealed that when the uncertainty of immigrants’ growth is more than a threshold level, increasing uncertainty of immigrants’ growth accelerates the optimal timing of firms’ investment and enhances the economic growth, even if the uncertainty of immigrants’ growth is formulated by the geometric Brownian motion, which is in sharp contrast to the standard result that an increase in the uncertainty postpones the optimal timing. With an increase in the immigrants over the past ten years, workforces in the host countries have been growing significantly to the extent that the immigrants represent 70% of the increase in the workforce in Europe, and 47% in the United States as OECD indicates. In the present paper, we attempted to investigate the effect of increased uncertainty caused by the growi...
发表评论
阅读全文

Education Policy Implementation: A Mechanism for Enhancing Primary Education Development in Zanzibar

Education is one of the fundamental rights of individuals; therefore, the government of a country needs to develop and strengthen educational policy and quality as well as to ensure that everyone has equal access to basic education. The improvement of access and quality of education in the world is becoming as an essential factor in development, whereas the basic education (primary school), is acknowledged as a foundation of the higher educational development for every country. To fulfill this goal, governments introduce several policies and procedures; however, it requires some reforms and participation from the politician, policymakers, and other stakeholders to re-examine educational policy so that it can lead to multiplication and betterment of the reforms. Educational reforms actually focus on accountability. A positive educational development and reform is very challenging and needs more effort and strategy on how to use and utilize the resources effectively as such it can achie...
发表评论
阅读全文