Text Classification Using Diffusion Kernel on Statistical Manifold
LI Kan1, ZHOU Shi-Bin2, LIU Yu-Shu1
School of Computer Science and Technology,Beijing Institute of Technology,Beijing 100081 School of Computer Science and Technology,China University of Mining and Technology,Xuzhou 221116
Abstract Dirichlet compound multinomial manifold (DCM manifold) is proposed. DCM manifold with positive sphere manifold is homeomorphic and isometric, so the geodesic distance of positive sphere manifold can be mapped as the geodesic distance of DCM manifold through pullback mapping. Then the distance metric is built on DCM manifold. DCM diffusion kernel function and DCMIDF diffusion kernel function are built on DCM manifold. The performance of the proposed algorithms for text classification are tested on the corpuses of WebKB Top 4 and 20 Newsgroups, and the experimental results show that DCM manifold is more desirable than that of Euclidean space in modeling texts on the corpuses. Compared with polynomial kernel based support vector machine and NGD kernel based support vector machine, the proposed DCM diffusion kernel and DCMIDF diffusion kernel based support vector machine algorithms show better computational accuracy for text classification.
[1] Rao C R.Information and Accuracy Attainable in the Estimation of Statistical Parameters.Bulletin of the Calcutta Mathematical Society,1945,37: 81-91 [2] encov N N.Statistical Decision Rules and Optimal Inference.Providence,USA: American Mathematical Society,1982: 477-493 [3] Campbell L L.An Extended C∨encov Characterization of the Information Metric.Proc of the American Mathematical Society,1986,98(1): 135-141 [4] Jaakkola T S,Haussler D.Exploiting Generative Models in Discriminative Classifiers // Kearns M S,Solla S A,Cohn D A,eds.Advances in Neural Information Processing Systems.Cambridge,USA: MIT Press,1999,XI: 487-493 [5] Jebara T,Kondor R,Honward A.Probability Product Kernels.Journal of Machine Learning Research,2004,5: 819-844 [6] Kondor R I,Lafferty J D.Diffusion Kernels on Graphs and Other Discrete Input Spaces // Proc of the 19th International Conference on Machine Learning.Edinburgh,Scotland,2002: 315-322 [7] Lafferty J,Lebanon G.Diffusion Kernels on Statistical Manifolds.Journal of Machine Learning Research,2004,6: 129-163 [8] Zhang D,Chen Xi,LEE W S.Text Classification with Kernels on the Multinomial Manifold // Proc of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.Salvador,Brazil,2005: 266-273 [9] Dollar P,Rabaud V,Belongie S.Non-Isometric Manifold Learning: Analysis and an Algorithm // Proc of the 24th International Conference on Machine Learning,Corvallis,USA,2007: 241-248 [10] Lin Tong,Zha Hongbin.Riemannian Manifold Learning.IEEE Trans on Pattern Analysis and Machine Intelligence,2008,30(5): 796-809 [11] Zhou Shibin.Text Categorization Based on Generative and Discriminative Model.Ph.D Dissertation.Beijing,China: Beijing Institute of Technology,2009 (in Chinese) (周世斌.基于生成模型和判别模型的文本分类技术研究.博士学位论文.北京:北京理工大学,2009) [12] Madsen R E,Kauchak D,Elkan C.Modeling Word Burstiness Using the Dirichlet Distribution // Proc of the 22nd International Conference on Machine Learning.Bonn,Germany,2005: 545-552 [13] Belkin M.Problems of Learning on Manifolds.Ph.D Dissertation.Chicago,USA: University of Chicago,2003 [14] Li Kan,Liu Yushu.Research on Noise Insensitive SVM Based Multi-Class Classification // Proc of International Conference on Machine Learning and Cybernetics.Shanghai,China,2004: 3234-3237
2012, Vol. 25 
