Abstract：Tree-augmented Nave bayes (TAN) classifier is a compromise between model complexity and classification rate. It is a hot research topic currently. To improve the classification accuracy of TAN classifier, a discriminative method based on Kullback-Leibler (KL) divergence is proposed. Explaining away residual (EAR) method is used to learn the structure of TAN, and then the TAN parameters are obtained by an objective function based on KL divergence. The experimental results on benchmark datasets show that the proposed method can get relatively high classification rates.
[1] Duda R, Hart P. Pattern Classification and Scene Analysis. New York, USA: John Wiley & Sons, 1973 [2] Friedman N, Geiger D, Goldszmidt M. Bayesian Network Classifiers. Machine Learning, 1997, 29(2/3): 131-163 [3] Greiner R, Su Xiaoyuan, Shen Bin, et al. Structural Extension to Logistic Regression: Discriminative Parameter Learning of Belief Net Classifiers. Machine Learning, 2005, 59(3): 297-322 [4] Grossman D, Domingos P. Learning Bayesian Network Classifiers by Maximizing Conditional Likelihood // Proc of the 21st International Conference on Machine Learning. Banff, Canada, 2004: 361-368 [5] Cooper G F, Herskovits E. A Bayesian Method for the Induction of Probabilistic Networks from Data. Machine Learning, 1992, 9(4): 309-347 [6] Pernkopf F, Bilmes J. Discriminative versus Generative Parameter and Structure Learning of Bayesian Network Classifiers // Proc of the 22nd International Conference on Machine Learning. Bonn, Germany, 2005: 657-664 [7] Han J W, Kamber M. Data Mining: Concepts and Techniques. Seattle, USA: Morgan Kaufmann, 2001 [8] Mitchell T M. Machine Learning. New York, USA: McGraw-Hill, 1997 [9] Chow C K, Liu C N. Approximating Discrete Probability Distributions with Dependence Trees. IEEE Trans on Information Theory, 1968, 14(3): 462-467 [10] Bilmes J A. Dynamic Bayesian Multinets // Proc of the 16th Conference on Uncertainty in Artificial Intelligence. Stanford, USA, 2000: 38-45 [11] Bilmes J A. Natural Statistical Models for Automatic Speech Recognition. Ph.D Dissertation. Berkeley, USA: University of California. Department of Electrical Engineering and Computer Science, 1999 [12] Shen Shiyi, Wu Zhonghua. Information Elements and Applications. Beijing, China: Higher Education Press, 2004 (in Chinese) (沈世镒,吴忠华.信息论基础与应用.北京:高等教育出版社, 2004) [13] Huang Kaizhu. Discriminative Nave Bayesian Classifiers [EB/OL]. [2005-10-01]. http://www.cse.cunk.edu.hk/~lyu/student/phd/kzhuang/03spring.pdf [14] Witten I H, Frank E. Data Ming: Practical Machine Learning Tools and Techniques with Java Implementations. Seattle, USA: Morgan Kaufmann, 2000: 265-314
2008, Vol. 21 
