Abstract The property of sample confidence measure function applied by ensemble algorithm of reducing noises is firstly analysed in this paper, and the reason of this function being unfit for multiclass dataset is expounded. Then a confidence measure function with more pertinence is designed, and an enhanced algorithm for reducing noises and ensemble parameters is proposed based on this function. Thus the discriminative parameters learning algorithm of Bayesian network not only effectively restrains the noise impact, but also avoids over fitting of classifiers, and further extend the application of discriminative Bayesian network calssifier applying ensemble learning algorithm in multiclass problem. Finally, the experimental results and its analysis on statistical hypothesis test verify that this algorithm more notably improves the classifier performance than ensemble parameters learning algorithms of Bayesian network at present.
WANG Zhong-Feng,WANG Zhi-Hai,FU Bin. A Smoothed Boosting Algorithm for Ensemble Parameters Learning of Bayesian Network Classifiers[J]. , 2010, 23(4): 508-515.
[1] Friedman N, Geiger D, Goldszmidt M. Bayesian Network Classifiers. Machine Learning, 1997, 29(2/3): 131-163 [2] Heckerman D. A Tutorial on Learning with Bayesian Networks. Cambridge, USA: MIT Press, 1999 [3] Chickering D M, Heckerman D, Meek C. Large-Sample Learning of Bayesian Networks is NP-Hard. The Journal of Machine Learning Research, 2004, 5: 1287-1330 [4] Keogh E J, Pazzani M J. Learning the Structure of Augmented Bayesian Classifiers. International Journal on Artificial Intelligence Tools, 2002, 11(4): 587-601 [5] Cerquides J, de Mántaras R L. TAN Classifiers Based on Decomposable Distributions. Machine Learning, 2005, 59(3): 323-354 [6] Tsamardinos I, Brown L E, Aliferis C F. The Max-Min Hill-Climbing Bayesian Network Structure Learning Algorithm. Machine Learning, 2006, 65(1): 31-78 [7] Pifer A C, Guedes L A. Structural Learning of Bayesian Networks Using a Modified MDL Score Metric. IEEE Latin America Transactions, 2007, 5(8): 644-651 [8] Heckerman D, Geiger D, Chickering D M. Learning Bayesian Networks: The Combination of Knowledge and Statistical Data. Machine Learning, 1995, 20(3): 197-243 [9] de Campos L M. A Scoring Function for Learning Bayesian Networks Based on Mutual Information and Conditional Independence Tests. The Journal of Machine Learning Research, 2006, 7: 2149-2187 [10] Greiner R, Zhou Wei. Structural Extension to Logistic Regression: Discriminative Parameter Learning of Belief Net Classifiers // Proc of the 18th Annual National Conference on Artificial Intelligence. Edmonton, Canada, 2002: 167-173 [11] Greiner R, Su Xiaoyuan, Shen Bin, et al. Structural Extension to Logistic Regression: Discriminative Parameter Learning of Belief Net Classifiers. Machine Learning, 2005, 59(3): 297-322 [12] Jing Yushi, Pavlovic＇ V, Rehg J M. Efficient Discriminative Learning of Bayesian Network Classifier via Boosted Augmented Nave Bayes // Proc of the 22nd International Conference on Machine Learning. Bonn, Germany, 2005: 369-376 [13] Jing Yushi, Pavlovic＇ V, Rehg J M. Boosted Bayesian Network Classifiers. Machine Learning, 2008, 73(2): 155-184 [14] Jin Rong, Zhang Jian. A Smoothed Boosting Algorithm Using Probabilistic Output Codes // Proc of the 22nd International Conference on Machine Learning. New York, USA, 2005: 361-368 [15] Jin Rong, Zhang Jian. Multi-Class Learning by Smoothed Boosting. Machine Learning, 2007, 67(3): 207-227 [16] Friedman J, Hastie T, Tibshirani R. Additive Logistic Regression: A Statistical View of Boosting. The Annals of Statistics, 2000, 28(2): 337-374 [17] Witten I H, Frank E. Data Mining: Practical Machine Learning Tools and Techniques. San Francisco, USA: Morgan Kaufmann, 2005 [18] Asuncion A, Newman D J. UCI Machine Learning Repository [DB/OL]. [2009-04-03]. http://www.ics.uci.edu/~mlearn/MLRepository.html
2010, Vol. 23 
