A Sparse Least Squares Support Vector Machine Classifier
LIU XiaoMao1, KONG Bo1, GAO JunBin2, ZHANG Jun3
1.Department of Mathematics, Huazhong University of Science and Technology, Wuhan 430074 2.School of Information Technology, Charles Sturt University, Bathurst, NSW 2795, Australia 3.State Key Laboratory for MultiSpectral Information Processing Technologies, Huazhong University of Science and Technology, Wuhan 430074
Abstract Support Vector Machine (SVM) has to solve the quadratic programming problem, while least squares support vector machine (LSSVM) only needs to deal with the linear equations. However the defect of LSSVM is the lack of sparseness. In this paper, a method named sparse least squares support vector machine classifier (SLSSVM) is presented to remedy the defect of the LSSVM. It is carried out by preextracting margin vectors using center distance ratio method as original training samples and putting those which have not been classified correctly in the first training together as new training samples. The proposed method not only remedies the defect of LSSVM, but also speeds up training and classifying. Furthermore, it can rectify the deviation of the classifier for unbalanced training data and the classifying ability is not affected. The good performance of SLSSVM is verified on several data sets.
[1] Vapnik V N. The Nature of Statistical Learning Theory. Berlin, Germany: SpringerVerlag, 1995 (Vapnik V N.统计学习理论.许建华,张学工,译.北京:电子工业出版社, 2004) [2] Deng Naiyang, Tian Yingjie. Support Vector Machines: New Methods in Data Mining. Beijing, China: Science Press, 2004 (in Chinese) (邓乃扬,田英杰.数据挖掘中的新方法——支持向量机.北京:科学出版社, 2004) [3] Xue Yi. Support Vector Machine and Math Programming. Ph.D Dissertation. Beijing, China: Beijing University of Technology. College of Applied Sciences, 2003 (in Chinese) (薛 毅.支持向量机与数学规划.博士学位论文.北京:北京工业大学.应用数理学院, 2003) [4] Suykens J A K, Vandewalle J. Least Squares Support Vector Machine Classifiers. Neural Process Letters, 1999, 9(3): 293300 [5] Suykens J A K, Lukas L, Vandewalle J. Sparse Least Squares Support Vector Machine Classifiers // Proc of the European Symposium on Artificial Neural Networks. Bruges, Belgium, 2000: 3742 [6] Zhang Li, Zhou Weida, Jiao Licheng. PreExtracting Support Vectors for Support Vector Machine // Proc of the 5th International Conference on Signal Processing. Beijing, China, 2000, Ⅱ: 14321435 [7] Zhang Li. Support Vector Machine and Kernel. Ph.D Dissertation. Xi’an, China: Xidian University. Key Laboratory of Radar Signal Processing, 2002 (in Chinese) (张 莉.支撑矢量机与核方法研究.博士学位论文.西安:西安电子科技大学.雷达信号处理重点实验室, 2002) [8] Cortes C, Vapnik N. Support Vector Networks. Machine Learning, 1995, 20(3): 273297 [9] Domeniconi C, Gunopulos D. Incremental Support Vector Machine Construction // Proc of IEEE International Conference on Data Mining. San Jose, USA, 2001: 589593 [10] Blake C L, Merz C J. UCI Repository of Machine Learning Databases [DB/OL]. [20051226]. http://www.ics.uci.edu/~mlearn/databases
2007, Vol. 20 
