SVM Parameter Selection Algorithm Based on Maximum Kernel Similarity Diversity
TANG Yao-Hua1,GUO Wei-Min1,GAO Jing-Huai2
1.Thermal Power Institute,Henan Electric Power Research Institute,Zhengzhou 450052 2.School of Electronic and Information Engineering,Xian Jiaotong University,Xian 710049
Abstract Aiming at support vector machine (SVM) parameter selection problem, a novel Gaussian kernel parameter rapid selection algorithm is proposed on the basis of kernel similarity diversity maximum (MSD) by analyzing the equivalent network model and the classification principle of SVM. In addition, MSD is combined with parameter search algorithm based on cross validation, and thus it is a composite parameter selection algorithm (MSD-GS) to the realize rapid and optimal selection of kernel parameter and regularization parameter. Simulation experiment results on data sets from UCI show that MSD-GS has the merits of simpleness, celerity and accurate parameter selection with no need of adding prior knowledge. The parameter selection result is better than the traversing exponential grid search algorithm. The selected couple of SVM parameters can make SVM get high generalization performance.
[1] Chapelle O, Vapnik V, Bousqet O, et al. Choosing Multiple Parameters for Support Vector Machines. Machine Learning, 2002, 46(1): 131-159 [2] Cristianini N, Campbell C, Taylor J S. Dynamically Adapting Kernels in Support Vector Machines // Proc of the Neural Information Processing Workshop. Breckenridge, USA, 1999, Ⅱ: 204-210 [3] Larsen J, Svarer C, Andersen L N, et al. Adaptive Regularization in Neural Network Modeling // Orr G B, Müller K R, eds. Neural Networks: Trick of the Trade. Berlin, Germany: Springer, 1998: 113-132 [4] Bengio Y. Gradient-Based Optimization of Hyper-Parameters. Neural Computation, 2000, 12(8): 1889-1900 [5] Frhlich H, Zell A. Efficient Parameter Selection for Support Vector Machines in Classification and Regression via Model-Based Global Optimization // Proc of the International Joint Conference on Neural Networks. Montréal, Canada, 2005, Ⅲ: 1431-1436 [6] Hsu C W, Chang C C, Lin C J. A Practical Guide to Support Vector Classification [EB/OL]. [2007-03-10]. http://www.csie.ntu.edu.tw/cjlin/papers/guide/guide.pdf [7] Huang C M, Lee Y J, Lin D K J, et al. Model Selection for Support Vector Machines via Uniform Design. Computational Statistics and Data Analysis, 2007, 52(1): 335-346 [8] Smola A J, Schlkopf B. A Tutorial on Support Vector Regression. Statistics and Computing, 2004, 14(3): 199-222 [9] Louw N, Steel S J. Variable Selection in Kernel Fisher Discriminate Analysis by Means of Recursive Feature Elimination. Computational Statistics and Data Analysis, 2006, 51(3): 2043-2055 [10] Blake C L, Merz C J. UCI Repository of Machine Learning Databases [DB/OL]. [2007-02-15]. Irvine, USA: http://www.ics.uci.edu/mlearn/MLRepository.html
2010, Vol. 23 
