Regression Analysis for Functional Data Based on Least Squares Support Vector Machine
MENG Yin-Feng1,2, LIANG Ji-Ye1,3
1.School of Computer and Information Technology, Shanxi University, Taiyuan 030006 2.School of Mathematical Sciences, Shanxi University, Taiyuan 030006. 3.Computer Science and Technology Department, Taiyuan Normal University, Taiyuan 030012
Abstract Partial functional linear model is used to explore the relationship between the mixed-type input containing a functional variable and a numerical vector and a numerical output. To improve the accuracy of prediction, based on the representation of the functional coefficient in reproducing kernel Hilbert space, the structured representation of the model is obtained. The estimation problem of the functional coefficient is converted into the estimation problem of a parameter vector, and the least squares support vector machine method is used for parameter estimation. Experimental results show that the performance of vector coefficient estimator is similar to other parameter estimation methods while the functional coefficient estimator is stabler and more accurate than the others, and the good performance of the proposed method further ensures the accuracy of machine learning.
[1] Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning-Data Mining, Inference, and Prediction. 2nd Edition. New York, USA: Springer, 2009 [2] Ramsay J O, Silverman B W. Functional Data Analysis. 2nd Edition. New York, USA: Springer, 1997 [3] Staicu A M, Crainiceanu C M, Carroll R J. Fast Methods for Spatially Correlated Multilevel Functional Data. Biostatistics, 2010, 11 (2): 177-194 [4] Zhang D W, Lin X H, Sowers M F. Two-Stage Functional Mixed Models for Evaluating the Effect of Longitudinal Covariate Profiles on a Scalar Outcome. Biometrics, 2007, 63(2): 351-362 [5] Shin H J. Partial Functional Linear Regression. Journal of Statistical Planning and Inference, 2009, 139(10): 3405-3418 [6] Zhou J J, Chen M. Spline Estimators for Semi-functional Linear Model. Statistics and Probability Letters, 2012, 82(3): 505-513 [7] Lee H J. Functional Data Analysis: Classification and Regression [EB/OL]. [2013-10-20]. http://repository.tamu.edu/bitstream/handle/1969.1/2805/etd-tamu-2004B-STAT-Lee.pdf?sequence=1 [8] Suykens J A K, Vandewalle J. Least Squares Support Vector Machine Classifiers. Neural Processing Letters, 1999, 9(3): 293-300 [9] Espinoza M, Suykens J A K, de Moor B. Kernel Based Partially Linear Models and Nonlinear Identification. IEEE Trans on Automatic Control, 2005, 50(10): 1602-1606 [10] Espinoza M, Suykens J A K, de Moor B. Partially Linear Models and Least Squares Support Vector Machines // Proc of the 43rd IEEE Conference on Decision and Control. Atlantis, USA, 2004, IV: 3388-3393 [11] Suykens J A K, Alzate C, Pelckmans K. Primal and Dual Model Representations in Kernel-Based Learning. Statistics Surveys, 2010, 4: 148-183 [12] Luts J, Molenberghs G, Verbeke G, et al. A Mixed Effects Least Squares Support Vector Machine Model for Classification of Longitudinal Data. Computational Statistics and Data Analysis, 2012, 56(3): 611-628 [13] Adams R A, Fournier J J F. Sobolev Spaces. 2nd Edition. Amsterdam, the Netherlands: Academic Press, 2003 [14] Wahba G. Support Vector Machines, Reproducing Kernel Hilbert Spaces, and the Randomized GACV // Schlkopf B, Burges C J C, Smola A J, eds. Advances in Kernel Methods-Support Vector Learning. Cambridge, USA: MIT Press, 1999: 69-88 [15] Wahba G. Spline Models for Observational Data. Philadelphia, USA: Society for Industrial and Applied Mathematics, 1990 [16] Vapnik V N. The Nature of Statistical Learning Theory. New York, USA: Springer-Verlag, 1995 [17] Vapnik V N. Statistical Learning Theory. New York, USA: John Wiley & Sons, 1998 [18] Vapnik V N. The Support Vector Method of Function Estimation // Suykens J A K, Vandewalle J, eds. Nonlinear Modeling: Advanced Black-Box Techniques. Boston, UK: Kluwer Academic Publishers, 1998: 55-85 [19] Yuan X M, Yang M, Yang Y. An Ensemble Classifier Based on Structural Support Vector Machine for Imbalanced Data. Pattern Recognition and Artificial Intelligence, 2013, 26(3): 315-320 (in Chinese) (袁兴梅,杨 明,杨 杨.一种面向不平衡数据的结构化SVM 集成分类器.模式识别与人工智能, 2013, 26(3): 315-320) [20] Guo H S, Wang W J. Dynamical Granular Support Vector Regression Machine. Journal of Software, 2013, 24(11): 2535-2547 (in Chinese) (郭虎升,王文剑.动态粒度支持向量回归机.软件学报, 2013, 24(11): 2535-2547) [21] Smola A J, Schlkopf B. A Tutorial on Support Vector Regression. Statistics and Computing, 2004, 14(3): 199-222 [22] Ruppert D. Selecting the Number of Knots for Penalized Splines. Journal of Computational and Graphical Statistics, 2002, 11(4): 735-757
2014, Vol. 27 
