Estimation of Posterior Probability and Applications: An Approach Based on Kernel Logistic Regression
LI Tao1, WANG JunPu1, WU XiuQing2, TANG JinHui2
1.Department of Automation, University of Science and Technology of China, Hefei 230027 2.Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei 230027
Abstract：A method based on feature vector set is proposed to render a sparse solution for kernel Logistic Regression (LR) and decrease computation complexity of posterior probability estimation. The proposed method is combined with Markov Random Field (MRF) in terms of Bayes rule, in which the conditional probability is replaced with posterior probability estimated by kernel LR. The combination is applied to image segmentation. Experiments on texture image segmentation show the performance of the proposed method is suporior to that of Gaussian MRF method.
李滔，王俊普，吴秀清，唐金辉. 后验概率估计及其应用:基于核Logistic回归的方法*[J]. 模式识别与人工智能, 2006, 19(6): 689-695.
LI Tao, WANG JunPu, WU XiuQing, TANG JinHui. Estimation of Posterior Probability and Applications: An Approach Based on Kernel Logistic Regression. , 2006, 19(6): 689-695.
[1] Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning: Data Mining, Inference and Prediction. Berlin, Germany: Springer, 2001 (Hastie T, Tibshirani R, Friedman J. 统计学习基础:数据挖掘,推理与预测.范 明,译.北京:电子工业出版社, 2004) [2] Jaakkola T S, Haussler D. Probabilistic Kernel Regression Models // Proc of the 7th International Workshop on Artificial Intelligence and Statistics. San Francisco, USA: Morgan Kaufmann, 1999: 99-108 [3] Roth V. Probabilistic Discriminative Kernel Classifiers for Multi-Class Problems // Radig B, Florczyk S, eds. Lecture Notes in Computer Science. London, UK: Springer-Verlag, 2001, 2191: 246-253 [4] Vapnik V N. The Nature of Statistical Learning Theory. New York, USA: Spinger, 1995 (Vapnik V N.统计学习理论的本质.张学工,译.北京:清华大学出版社, 2000) [5] Friedman J, Hastie T, Tibshirani R. Additive Logistic Regression: A Statistical View of Boosting. Annals of Statistics, 2000, 28(2): 337-407 [6] Roth V, Steinhage V. Nonlinear Discriminant Analysis Using Kernel Functions // Solla S A, Leen T K, Muller K R, eds. Advances in Neural Information Processing Systems. Cambridge, USA: MIT Press, 1999, 12: 568-574 [7] Gao F, Klein R, Klein B, et al. Smoothing Spline ANOVA Models for Large Data Sets with Bernoulli Observations and the Randomized GACV. Annals of Statistics, 2000, 28(6):1570-1600 [8] Smola A J, Schlkopf B. Sparse Greedy Matrix Approximation for Machine Learning // Langley P, ed. Proc of the 17th International Conference on Machine Learning. San Francisco, USA: Morgan Kaufmann, 2000: 911-918 [9] Williams C K I, Seeger M. Using the Nystrom Method to Speed up Kernel Machines // Leen T K, Dietterich T G, Tresp V, eds. Advances in Neural Information Processing Systems. Cambridge, USA: MIT Press, 2001, 13: 682-688 [10]Baudat G, Anouar F. Feature Vector Selection and Projection Using Kernels. Neurocomputing, 2003, 55(1/2): 21-38 [11]Minka T. Algorithms for Maximum-Likelihood Logistic Regression. Technical Report, 758. Pittsburgh, USA: Carnegie Mellon University. Department of Statistics, 2001 [12]Rtsch G, Onoda T, Müller K R. Soft Margins for AdaBoost. Journal of Machine Learning, 2001, 42(3): 287-320 [13]Hastie T, Tibshirani R. Classification by Pairwise Coupling. Annals of Statistics, 1998, 26(2): 451-471 [14]Besag J. On the Statistical Analysis of Dirty Pictures. Journal of the Royal Statistical Society: Series B (Methodological), 1986, 48(3): 259-302 [15]Li S Z. Markov Random Field Modeling in Computer Vision. Berlin, Germany: Springer-Verlag, 1995 [16]Giordana N, Pieczynski W. Estimation of Generalized Multisensor Hidden Markov Chains and Unsupervised Image Segmentation. IEEE Trans on Pattern Analysis and Machine Intelligence, 1997, 19(5): 465-475 [17]Haralick R. Statistical and Structural Approaches to Texture. Proc of the IEEE, 1979, 67(5): 786-804 [18]Berthod M, Kato Z, Shan Y, et al. Bayesian Image Classification Using Markov Random Fields. Image and Vision Computing, 1996, 14(4): 285-295
2006, Vol. 19 
