Multi-class Classification Algorithm Based on Ensemble Learning and Hierarchical Structure
ZOU Quan1,2,3, SONG Li1, CHEN Wen-Qiang1 , ZENG Jian-Cang1, LIN Chen1,2
1.School of Information Science and Engineering, Xiamen University, Xiamen 361005 2.Shenzhen Research Institute, Xiamen University, Shenzhen 518057 3.School of Computer Science and Technology, Tianjin University, Tianjin 300072
Abstract The classification algorithm is an important research field in data mining and pattern recognition. A multi-class classification algorithm based on ensemble learning and hierarchical structure is proposed. Firstly, the problems are decomposed according to their hierarchy categories. The hierarchical structure of the hierarchical classifier is defined. Then, multiple weak classifiers are integrated by ensemble learning methods based on the hierarchical structure. Thus, the classification process is completed. In the data mining competition of CCDM 2014, the proposed algorithm achieves the highest performance on several indexes, including average accuracy and F1-score. The results verify the feasibility on the classification problem.
ZOU Quan,SONG Li,CHEN Wen-Qiang等. Multi-class Classification Algorithm Based on Ensemble Learning and Hierarchical Structure[J]. , 2015, 28(9): 781-787.
[1] Cortes C, Vapnik V. Support-Vector Networks. Machine Learning, 1995, 20(3): 273-297 [2] Safavian S R, Landgrebe D. A Survey of Decision Tree Classifier Methodology. IEEE Trans on Systems, Man, and Cybernetics, 1991, 21(3): 660-674 [3] Caruana R, Niculescu-Mizil A, Crew G, et al. Ensemble Selection from Libraries of Models // Proc of the 21st International Conference on Machine Learning. Banff, Canada, 2004: 137-144 [4] Zhou Z H, Wu J X, Tang W. Ensembling Neural Networks: Many Could Be Better Than All. Artificial Intelligence, 2002, 137(1/2): 239-263 [5] Zhang C X, Zhang J S. A Survey of Selective Ensemble Learning Algorithm. Chinese Journal of Computers, 2011, 34(8): 1399-1410 (in Chinese) (张春霞,张讲社.选择性集成学习算法综述.计算机学报, 2011, 34(8): 1399-1410) [6] Breiman L. Bagging Predictors. Machine Learning, 1996, 24(2): 123-140 [7] Freund Y, Schapire R E. A Decision-Theoretic Generalization of On-line Learning and an Application to Boosting. Journal of Computer and System Sciences, 1997, 55(1): 119-139 [8] Kearns M J, Valiant L G. Learning Boolean Formulae or Finite Automata is as Hard as Factoring. Technical Report, TR-14-88. Cambridge, USA: Harvard University, 1988 [9] Breiman L. Random Forests. Machine Learning, 2001, 45(1): 5-32 [10] Friedman J H. Stochastic Gradient Boosting. Computational Statistics & Data Analysis, 2002, 38(4): 367-378 [11] Friedman J H. Greedy Function Approximation: A Gradient Boo-sting Machine. The Annals of Statistics, 2001, 29(5): 1189-1232 [12] Wolpert D H. The Lack of a Priori Distinctions between Learning Algorithms. Neural Computation, 1996, 8(7): 1341-1390 [13] Dietterich T G. Ensemble Methods in Machine Learning // Proc of the 1st International Workshop on Multiple Classifier Systems. Cagliari, Italy, 2000: 1-15 [14] Dietterich T G, Bakiri G. Solving Multiclass Learning Problems via Error-Correcting Output Codes. Journal of Artificial Intelligence Research, 1995, 2: 263-286 [15] Tsoumakas G, Katakis I, Vlahavas I. Mining Multi-label Data // Maimon O, Rokach L, eds. Data Mining and Knowledge Discovery Handbook. New York, USA: Springer, 2010: 667-685 [16] Lin C, Chen W Q, Qiu C, et al. LibD3C: Ensemble Classifiers with a Clustering and Dynamic Selection Strategy. Neurocomputing, 2014, 123: 424-435 [17] Rendle S. Factorization Machines // Proc of the 10th IEEE International Conference on Data Mining. Sydney, Australia, 2010: 995-1000 [18] Rendle S. Factorization Machines with LibFM. ACM Trans on Intelligent Systems and Technology, 2012. DOI: 10.1145/2168752.2168771 [19] Zhang M L, Zhou Z H. Multi-label Learning by Instance Differentiation // Proc of the 22nd AAAI Conference on Artificial Intelligence. Vancouver, Canada, 2007: 669-674
2015, Vol. 28 
