Support Tensor Machine Classifier with Pinball Loss
YU Keming1, HAN Le1, YANG Xiaowei2
1.School of Mathematics, South China University of Technology, Guangzhou 510640 2.School of Software Engineering, South China University of Technology, Guangzhou 510006
Abstract The input patterns are usually high-order tensors in the fields of machine learning, pattern recognition, data mining, etc. In this paper, the pin-support vector machine is firstly extended from vector to tensor and the support tensor machine (STM) classifier with pinball loss(pin-STM) is proposed. Then, a sequential minimal optimization (SMO) algorithm is designed to solve this model. To maintain the nature structure of tensor and speed up the training procedure, the rank-one decomposition of tensor is used to substitute the original tensor to compute the inner products of tensors. The experimental results on vector datasets and tensor datasets show that SMO is faster than the classical active-set method for vector data. Compared with pin-SVM, the pin-STM has higher training speed and better generalized performance for tensor data.
About author:: YU Keming, born in 1991, master student. Her research interests include machine learning and tensor analysis.HAN Le, born in 1977, Ph.D., associate professor. Her research interests include matrix optimization and machine lear-ning.YANG Xiaowei(Corresponding author), born in 1969, Ph.D., professor. His research interests include machine learning, pattern recognition, data mining and tensor analysis.)
[1] LIN C F, WANG S D. Fuzzy Support Vector Machines. IEEE Trans on Neural Networks, 2002, 13(2): 464-471. [2] LIN C F, WANG S D. Training Algorithms for Fuzzy Support Vector Machines with Noisy Data. Pattern Recognition Letters, 2004, 25(14): 1647-1656. [3] JAYADEVA, KHEMCHANDANI R, CHANDRA S. Fast and Robust Learning through Fuzzy Linear Proximal Support Vector Machines. Neurocomputing, 2004, 61: 401-411. [4] WANG Y Q, WANG S Y, LAI K K. A New Fuzzy Support Vector Machine to Evaluate Credit Risk. IEEE Trans on Fuzzy Systems, 2005, 13(6): 820-831. [5] BATUWITA R, PALADE V. FSVM-CIL: Fuzzy Support Vector Machines for Class Imbalance Learning. IEEE Trans on Fuzzy Systems, 2010, 18(3): 558-571. [6] YANG X W, ZHANG G Q, LU J, et al. A Kernel Fuzzy C-means Clustering-Based Fuzzy Support Vector Machine Algorithm for Classification Problems with Outliers or Noises. IEEE Trans on Fuzzy Systems, 2010, 19(1): 105-115. [7] SUYKENS J A K, DE BRABANTER J, LUKAS L, et al. Weighted Least Squares Support Vector Machines: Robustness and Sparse Approximation. Neurocomputing, 2002, 48(1/2/3/4): 85-105. [8] ZHANG X G. Using Class-Center Vectors to Build Support Vector Machines // Proc of the IEEE Signal Processing Society Workshop on Neural Networks for Signal Processing. Madison, USA: IEEE, 1999: 3-11. [9] TAO Q, WANG J. A New Fuzzy Support Vector Machine Based on the Weighted Margin. Neural Processing Letter, 2004, 20(3): 139-150. [10] SONG Q, HU W J, XIE W F. Robust Support Vector Machine with Bullet Hole Image Classification. IEEE Trans on Systems, Man, and Cybernetics(Applications and Reviews), 2002, 32(4): 440-448. [11] XU L L, CRAMMER K, SCHUURMANS D. Robust Support Vector Machine Training via Convex Outlier Ablation [C/OL].[2015-06-18]. http://webdocs.cs.ualberta.ca/~dale/papers/aaai06a.pdf. [12] WU Y C, LIU Y F. Robust Truncated Hinge Loss Support Vector Machines. Journal of the American Statistical Association, 2007, 102(479): 974-983. [13] YANG X W, TAN L J, HE L F. A Robust Least Squares Support Vector Machine for Regression and Classification with Noise. Neurocomputing, 2014, 140: 41-52. [14] YANG X W, HAN L, LI Y, et al. A Bilateral-Truncated-Loss Based Robust Support Vector Machine for Classification Problems. Soft Computing, 2015, 19(10): 2871-2882. [15] HUANG X L, SHI L, SUYKENS J A K. Support Vector Machine Classifier with Pinball Loss. IEEE Trans on Pattern Analysis and Machine Intelligence, 2013, 36(5): 984-997. [16] CAI D, HE X F, WEN J R, et al. Support Tensor Machines for Text Categorization. Technical Report, UIUCDCS-R-2006-2714. Urbana, USA: University of Illinois at Urbana-Champaign, 2006. [17] WANG Z, CHEN S C. New Least Squares Support Vector Machines Based on Matrix Patterns. Neural Processing Letters, 2007, 26(1): 41-56. [18] TAO D C, LI X L, WU X D, et al. Supervised Tensor Learning. Knowledge and Information Systems, 2007, 13(1): 1-42. [19] WU F, LIU Y N, ZHUANG Y T. Tensor-Based Transductive Learning for Multimodality Video Semantic Concept Detection. IEEE Trans on Multimedia, 2009, 11(5): 868-878. [20] ZHANG X S, GAO X B, WANG Y. Twin Support Tensor Machines for MCs Detection. Journal of Electronics (China), 2009, 26(3): 318-325. [21] LIU Y, LIU Y, CHAN K C C. Tensor-Based Locally Maximum Margin Classifier for Image and Video Classification. Computer Vision and Image Understanding, 2011, 115(3): 300-309. [22] ZHANG L F, ZHANG L P, TAO D C, et al. A Multifeature Tensor for Remote-Sensing Target Recognition. IEEE Geoscience and Remote Sensing Letters, 2010, 8(2): 374-378. [23] SIGNORETTO M, DE LATHAUWER L, SUYKENS J A K. A Kernel-Based Framework to Tensorial Data Analysis. Neural Networks, 2011, 24(8): 861-874. [24] GUO W W, KOTSIA I, PATRAS I. Tensor Learning for Regression. IEEE Trans on Image Processing, 2012, 21(2): 816-827. [25] HOU C P, NIE F P, ZHANG C S, et al. Multiple Rank Multi-linear SVM for Matrix Data Classification. Pattern Recognition, 2014, 47(1): 454-469. [26] HAO Z F, HE L F, CHEN B Q, et al. A Linear Support Higher-Order Tensor Machine for Classification. IEEE Trans on Image Processing, 2013, 22(7): 2911-2920. [27] LU H P, PLATANIOTIS K N, VENETSANOPOULOS A N. MPCA: Multilinear Principal Component Analysis of Tensor Objects. IEEE Trans on Neural Networks, 2008, 19(1): 18-39. [28] KOLDA T G, BADER B W. Tensor Decompositions and Applications. SIAM Review, 2009, 51(3): 455-500. [29] NOCEDAL J, WRIGHT S J. Numerical Optimization. 2nd Edition. New York, USA: Springer-Verlag, 2006. [30] SHEVADE S K, KEERTHI S S, BHATTACHARYYA C, et al. Improvements to the SMO Algorithm for SVM Regression. IEEE Trans on Neural Networks, 2000, 11(5): 1188-1193. [31] KEERTHI S S, SHEVADE S K, BHATTACHARYYA C, et al. Improvements to Platt's SMO Algorithm for SVM Classifier Design. Neural Computation, 2001, 13(3): 637-649. [32] FLAKE G W, LAWRENCE S. Efficient SVM Regression Training with SMO. Machine Learning, 2002, 46(1): 271-290. [33] KIERS H A L. An Alternating Least Squares Algorithm for PARA- FAC2 and Three-Way DEDICOM. Computational Statistics & Data Analysis, 1993, 16(1): 103-118. [34] ZHANG T, GOLUB G H. Rank-One Approximation to High Order Tensors. SIAM Journal on Matrix Analysis and Applications, 2001, 23(2): 534-550. [35] RIPLEY B D. Pattern Recognition and Neural Networks. Cambr- idge, UK: Cambridge University Press, 1996.
[36] LIN C J. Asymptotic Convergence of an SMO Algorithm without Any Assumptions. IEEE Trans on Neural Networks, 2002, 13(1): 248-250
2016, Vol. 29 
