Smoothing Functions for Support Vector Regressions
XIONG Jin-Zhi1,2, HU Jin-Lian2, YUAN Hua-Qiang2, HU Tian-Ming2, PENG Hong1
1.School of Computer Science and Engineering, South China University of Technology,Guangzhou 5106412. Software College, Dongguan University of Technology, Dongguan 523808
Abstract Smoothing functions can transform the unsmooth support vector machines (SVMs) into smooth ones, and thus better regression results are generated. A smoothing function was used by Lee et al. to approximate the square ofε-insensitive loss function, therefore the ε-insensitive smooth support vector regression (ε-SSVR) was proposed. In this paper, using techniques of interpolation function and function composition, a kind of smoothing functions is proposed for ε-insensitive support vector regressions (SVRs). Smooth approximations of the plus function are firstly derived and then applied to approximate the square of the ε-insensitive loss function. Theoretical analysis shows that the approximation accuracy of the proposed smoothing functions is an order of magnitude higher than that of the existing ones. Better regression results are yielded and the new kind of smoothing functions is provided for SVRs.
[1] Chen Chunhui, Mangasarian O L. A Class of Smoothing Functions for Nonlinear and Mixed Complementarity Problems. Computational Optimization and Application, 1996, 5(2): 97-138 [2] Yuan Yubo, Yan Jie, Xu Chengxian. Polynomial Smooth Support Vector Machine (PSSVM). Chinese Journal of Computers, 2005, 28(1): 9-17 (in Chinese) (袁玉波,严 杰,徐成贤.多项式光滑的支撑向量机.计算机学报, 2005, 28(1): 9-17) [3] Chen Chunhui, Mangasarian O L. Smoothing Methods for Convex Inequalities and Linear Complementarity Problems. Mathematical Programming, 1995, 71(5): 51-69 [4] Mangasarian O L. Mathematical Programming in Neural Networks. ORSA Journal of Computing, 1993, 5(4): 349-360 [5] Lee Y J, Mangasarian O L. SSVM: A Smooth Support Vector Machine for Classification. Computational Optimization and Applications, 2001, 22(1): 5-21 [6] Lee Y J, Hsieh W F, Huang C M. ε-SSVR: A Smooth Support Vector Machine for ε-Insensitive Regression. IEEE Trans on Knowledge and Data Engineering, 2005, 17(5): 678-685 [7] Platt J. Fast Training of Support Vector Machines Using Sequential Minimal Optimization // Schlkopf B, Burges C, Smola A, eds. Advances in Kernel Methods: Support Vector Learning. Cambridge, USA: MIT Press, 1999: 557-563 [8] Joachims T. Making Large-Scale Support Vector Machine Learning Practical // Schlkopf B, Burges C, Smola A, eds. Advances in Kernel Methods: Support Vector Learning. Cambridge, USA: MIT Press, 1999: 169-180 [9] Mangasarian O L, Musicant D R. Successive Overrelaxation for Support Vector Machines. IEEE Trans on Neural Networks, 1999, 10(5): 1032-1037 [10] Xiong Jinzhi, Hu Jinlian, Yuan Huaqiang, et al. Research on a New Class of Functions for Smoothing Support Vector Machines. Acta Electronica Sinica, 2007, 35(2): 366-370 (in Chinese) (熊金志,胡金莲,袁华强,等.一类光滑支持向量机新函数的研究.电子学报, 2007, 35(2): 366-370) [11] Mangasarian O L, Musicant D R. Lagrangian Support Vector Machines. Journal of Machine Learning Research, 2001, 1(1): 161-177 [12] Lee Y J, Mangasarian O L. RSVM: Reduced Support Vector Machines // Proc of the 1st SIAM International Conference on Data Mining. Chicago, USA, 2001: 350-366 [13] Fung G M, Mangasarian O L. Proximal Support Vector Machine Classifiers // Proc of the 7th International Conference on Knowledge Discovery and Data Mining. San Francisco, USA, 2001: 77-86 [14] Li Qingyang, Wang Nengchao, Yi Dayi. Numerical Analysis. Beijing, China: Tsinghua University Press, 2001 (in Chinese) (李庆扬,王能超,易大义.数值分析.北京:清华大学出版社, 2001) [15] Deng Naiyang, Tian Yingjie. New Method in Data Mining: Support Vector Machine. Beijing, China: Science Press, 2004 (in Chinese) (邓乃扬,田英杰.数据挖掘的新方法——支持向量机.北京:科学出版社, 2004)
2008, Vol. 21 
