|
|
|
| Linear Hybrid of Generative and Discriminative Classifiers |
| SHI Hong-Bo, LIU Ya-Qin |
| School of Information Management,Shanxi University of Finance and Economics,Taiyuan 030031 |
|
|
|
|
Abstract The generative approaches and the discriminative approaches are two kinds of paradigms for solving classification problems. To exploit the advantages of these approaches, a linear hybrid of generative/discriminative model (LHGD) is proposed, and a learning algorithm of LHGD based on genetic algorithms (LHGD_GA) is designed. LHGD_GA regards hybrid parameter learning of the linear hybrid classification model as an optimization problem, and utilizes genetic algorithms to find the best hybrid parameters of linear hybrid classification model. The experimental results show that the linear hybrid generative/discriminative classifier is better than or similar to the better classifier of two base classifiers on most datasets.
|
|
Received: 05 September 2011
|
|
|
|
|
|
[1] Rubinstein Y D,Hastie T.Discriminative vs.Informative Learning // Proc of the 3rd International Conference on Knowledge Discovery and Data Mining.New Beach,USA,1997: 49-53
[2] Ng A Y,Jordan M I.On Discriminative vs.Generative Classifiers: A Comparison of Logistic Regression and Nave Bayes // Becker S,Thrun S,Obermayer K,eds.Advances in Neural Information Processing Systems.Cambridge,USA: MIT Press,2002,XV: 841-848
[3] Bishop C M,Lasserre J.Generative or Discriminative? Getting the Best of Both Worlds // Proc of the 8th World Meeting on Bayesian Statistics.Alicante,Spain,2007: 3-24
[4] Xue J H,Titterington D M.On the Generative-Discriminative Tradeoff Approach: Interpretation,Asymptotic Efficiency and Classification Performance.Computational Statistic and Data Analysis,2010,54(2): 438-451
[5] Xue J H,Titterington D M.Joint Discriminative-Generative Modeling Based on Statistical Tests for Classification.Pattern Recognition Letters,2010,31(9): 1048-1055
[6] Raina R,Shen Y,Ng A Y,et al.Classification with Hybrid Generative/Discriminative Models // Thrun S,Saul L K,Schlkopf B,eds.Advances in Neural Information Processing Systems.Cambridge,USA: MIT Press,2003,XVI: 545-552
[7] Fujino A,Ueda N,Saito K.A Hybrid Generative/Discriminative Approach to Text Classification with Additional Information.Information Processing and Management,2007,43(2): 379-392
[8] Jaakkola T S,Haussler D.Exploiting Generative Models in Discriminative Classifiers // Kearns M J,Solla S A,Cohn D A,eds.Advances in Neural Information Processing Systems.Cambridge,USA: MIT Press,1998,II: 487-493
[9] Tu Z W.Learning Generative Models via Discriminative Approaches // Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Minneapolis,USA,2007: 1-8
[10] Rubinstein Y D.Discriminative vs.Informative Learning.Ph.D Dissertation.Stanford,USA: Stanford University,1998
[11] Bouchard G,Triggs B.The Trade-off between Generative and Discriminative Classifiers // Proc of the 16th International Symposium on Computational Statistics.Prague,The Czech Republic,2004: 721-728
[12] McCallum A,Pal C,Druck G,et al.Multi-Conditional Learning: Generative/Discriminative Training for Clustering and Classification // Proc of the 20th International Conference on Artificial Intelligence.Boston,USA,2006: 433-439
[13] Lasserre J A,Bishop C M,Minka T P.Principled Hybrid of Generative and Discriminative Models // Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Miami,USA,2006: 87-94
[14] Druck G,Pal C,McCallum A.Semi-Supervised Classification with Hybrid Generative/Discriminative Methods // Proc of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.San Jose,USA,2007: 280-289
[15] Duda R,Hart P.Pattern Classification and Scene Analysis.New York,USA: John Wiley Sons,1973
[16] Friedman N,Geiger D,Goldszmidt M.Bayesian Network Classifiers.Machine Learning,1997,29 (2/3): 131-163
[17] Jing Y,Pavlovic V,Rehg J M.Boosted Bayesian Network Classifiers.Machine Learning,2008,73(1): 155-184
[18] Hand D J,Mannila H,Smyth P.Principles of Data Mining.Cambridge,USA: MIT Press,2001
[19] de Jong K.Learning with Genetic Algorithm: An Overview.Machine Learning,1988,3(2/3): 121-138
[20] Witten I H,Frank E.Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations.Seattle,USA: Morgan Kaufmann Publishers,2000 |
|
|
|
2012, Vol. 25 
