A Kernel Based Supervised Manifold Learning Algorithm
LI Jun-Bao1, PAN Jeng-Shyang2
1.Institute of Automatic Test and Control, Harbin Institute of Technology, Harbin 1500012. Department of Electronic Engineering, Kaohsiung University of Applied Sciences, Kaohsiung China
Abstract A kernel based supervised manifold learning method is presented to solve the problems on parameter selection with locality preserving projection and inability in nonlinear feature extraction, which is unresolved by the currently proposed manifold learning algorithm. The proposed algorithm is an improvement of locality preserving projection (LPP). The nearest neighbor graph is created with the class label information of the samples, and nonparametric similarity measure is used. The kernel method is used to solve the limitation problem of the nonlinear separability for LPP. The feasibility and effectivity of the proposed algorithm are testified on two databases.
[1] Belhumeur P N, Hespanha J P, Kriegman D J. Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection. IEEE Trans on Pattern Analysis and Machine Intelligence, 1997, 19(7): 711-720 [2] Batur A U, Hayes M H. Linear Subspace for Illumination Robust Face Recognition // Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Kauai Marriott, USA, 2001, Ⅱ: 296-301 [3] Roweis S T, Saul L K. Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science, 2000, 290(5500): 2323-2326 [4] Tenenbaum J B, de Silva V, Langford J C. A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science, 2000, 290(5500): 2319-2323 [5] Belkin M, Niyogi P. Laplacian Eigenmaps for Dimensionality Reduction and Data Representation. Neural Computation, 2003, 15(6): 1373-1396 [6] He Xiaofei, Yan Shuicheng, Hu Yuxiao, et al. Face Recognition Using Laplacianfaces. IEEE Trans on Pattern Analysis and Machine Intelligence, 2005, 27(3): 328-340 [7] Geng Xin, Zhan Dechuan, Zhou Zhihua. Supervised Nonlinear Dimensionality Reduction for Visualization and Classification. IEEE Trans on Systems, Man and Cybernetics, 2005, 35(6): 1098-1107 [8] Zhang Junping. Machine Learning and Its Applications. Beijing, China: Tsinghua University Press, 2006 (in Chinese) (张军平.机器学习及其应用.北京:清华大学出版社, 2006) [9] Schlkopf B, Smola A, Müller K R. Nonlinear Component Analysis as a Kernel Eigenvalue Problem. Neural Computation, 1998, 10(5): 1299-1319 [10] Liu Chengjun. Gabor-Based Kernel PCA with Fractional Power Polynomial Models for Face Recognition. IEEE Trans on Pattern Analysis and Machine Intelligence, 2004, 26(5): 572-581 [11] Mika S, Ratsch G, Weston J, et al. Fisher Discriminant Analysis with Kernels // Proc of the IEEE Signal Processing Society Workshop on Neural Networks for Signal Processing. Madison, USA, 1999: 41-48 [12] Yang Jian, Frangi A F, Yang Jingyu, et al. KPCA Plus LDA: A Complete Kernel Fisher Discriminant Framework for Feature Extraction and Recognition. IEEE Trans on Pattern Analysis and Machine Intelligence, 2005, 27(2): 230-244 [13] Vapnik V N. The Nature of Statistical Learning Theory. New York, USA: Springer-Verlag, 1995 [14] Jin Bo, Tang Y C, Zhang Yanqing. Support Vector Machines with Genetic Fuzzy Feature Transformation for Biomedical Data Classification. Information Sciences, 2007, 177(2): 476-489 [15] Müller K R, Mika S, Rtsch G, et al. An Introduction to Kernel-Based Learning Algorithms. IEEE Trans on Neural Networks, 2001, 12(2): 181-201 [16] Schlkopf B, Burges C, Smola A J. Advances in Kernel Methods: Support Vector Learning. Cambridge, USA: MIT Press, 1999 [17] Ruiz A, López-de-Teruel P E. Nonlinear Kernel-Based Statistical Pattern Analysis. IEEE Trans on Neural Networks, 2001, 12(1): 16-32 [18] Samaria F, Harter A C. Parameterisation of a Stochastic Model for Human Face Identification // Proc of the 2nd IEEE Workshop on Applications of Computer Vision. Sarasota, USA, 1994: 138-142 [19] Bengio Y, Palement J F, Vincent P. Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering // Thrun S, Saul L, Schlkopf B, eds. Advances in Neural Information Processing Systems. Cambridge, USA: MIT Press, 2004: 307-311
2008, Vol. 21 
