Abstract The Minimum Class Variance Support Vector Machine (MCVSVM) takes into consideration both the samples in the boundaries and the distribution of the classes. However, only the information in the non-null space of the within-class scatter matrix is utilized in the case of small sample size. To further improve the classification performance, in this paper the Null Space Classifier (NSC) which is rooted in the null space is first presented, then an Ensemble Classifier (EC) is proposed by fusing the MCVSVM and the NSC. Different form the MCVSVM and the NSC, the EC considers the information both in the non-null space and in the null space and has better generalizability. Finally, experimental results on several real datasets indicate the effectiveness of the EC.
[1] Vapnik V N. Statistical Learning Theory. New York, USA: Wiley, 1998 [2] Er M J, Chen W, Wu S. High-Speed Face Recognition Based on Discrete Cosine Transform and RBF Neural Networks. IEEE Trans on Neural Networks, 2005, 16(3): 679-691 [3] Tong S, Chang E. Support Vector Machine Active Learning for Image Retrieval // Proc of the ACM International Conference on Multimedia. Ottawa, Canada, 2001: 107-118 [4] Veeramachaneni S, Nagy G. Style Context with Second-Order Statistics. IEEE Trans on Pattern Analysis and Machine Intelligence, 2005, 27(1): 14-22 [5] Ganapathiraju A, Hamaker J E, Picone J. Applications of Support Vector Machines to Speech Recognition. IEEE Trans on Signal Processing, 2004, 52(8): 2348-2355 [6] Zafeiriou S, Tefas A, Pitas I. Minimum Class Variance Support Vector Machines. IEEE Trans on Image Processing, 2007, 16(10): 2551-2564 [7] Duda R O, Hart P E, Stork D G. Pattern Classification. 2nd Edition. New York, USA: Wiley, 2001 [8] Yang J, Yang J Y. Why Can LDA Be Performed in PCA Transformed Space? Pattern Recognition, 2003, 36(2): 563-566 [9] Zhang X X, Jia Y D. A Linear Discriminant Analysis Framework Based on Random Subspace for Face Recognition. Pattern Recognition, 2007, 40(9): 2585-2591 [10] Fung G, Mangasarian O L. Proximal Support Vector Machine Classifiers // Proc of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Francisco, USA, 2001: 77-86 [11] Gestel T V, Suykens J A K, Baesens B, et al. Benchmarking Least Squares Support Vector Machine Classifiers. Machine Learning, 2004, 54(1): 5-32 [12] Schlkopf B, Smola A. Learning with Kernels. Cambridge, USA: MIT Press, 2002 [13] Schlkopf B, Smola A, Muller K R. Nonlinear Component Analysis as a Kernel Eigenvalue Problem. Neural Computing, 1998, 10(5): 1299-1319 [14] Abril L G, Angulo C, Velasco F, et al. A Note on the Bias in SVMs for Multiclassification. IEEE Trans on Neural Networks, 2008, 19(4): 723-725 [15] Cai Deng, He Xiaofei, Han Jiawei, et al. Orthogonal Laplacianfaces for Face Recognition. IEEE Trans on Image Processing, 2006, 15(11): 3608-3614
2010, Vol. 23 
