An Improved PCA Algorithm with Local Structure Preserving
WANG Qing-Gang1, LI Jian-Wei1,2
1.Key Laboratory of Optoelectronic Technology and Systems of Ministry of Education,College of Optoelectronic Engineering, Chongqing University, Chongqing 400030 2.Chongqing University of Technology, Chongqing 400050
Abstract Locality preserving projection (LPP) is a local structure preserving method and the distances of neighboring points are minimized in the subspace of LPP. Combined with the geometric idea of LPP, an improved PCA with local structure preserving is proposed called locality preserving PCA (LP-PCA). By constructing the neighborhood graph and its complement, LP-PCA deals with the neighboring points and the far points distinguishingly. LP-PCA minimizes the distances between the neighboring points and simultaneously maximizes the distances between the far points. The improved algorithm can find the global structure of the high dimensional dataset with preserving its local structure. Some examples of the improved algorithm are given on toy datasets as well as on actual datasets. Experimental results show the effectiveness of LP-PCA.
[1] Jolliffe I. Principal Component Analysis. New York, USA: Springer-Verlag, 1989 [2] Luo Siwei, Zhao Lianwei. Manifold Learning Algorithms Based on Spectral Graph Theory. Journal of Computer Research and Development, 2006, 43(7): 1173-1179 (in Chinese) (罗四维,赵连伟.基于谱图理论的流形学习算法.计算机研究与发展, 2006, 43(7): 1173-1179) [3] Zhang Junping. Research on Some Problems in Manifold Learning // Wang Jue, Zhou Zhihua, Zhou Aoying, eds. Machine Learning and Applications. Beijing, China: Tsinghua University Press, 2006: 135-169 (in Chinese) (张军平. 流形学习若干问题研究 // 王 珏,周志华,周傲英,编.机器学习及其应用.北京:清华大学出版社, 2006: 135-169) [4] He Xiaofei, Niyogi P. Locality Preserving Projections // Thrun S, Saul L K, Schlkopf B, eds. Advances in Neural Information Processing Systems. Cambridge, USA: MIT Press, 2003, 16: 56-87 [5] He Xiaofei. Locality Preserving Projections. Ph.D Dissertation. Illinois, USA: The University of Chicago. Department of Computer Science, 2005 [6] Belkin M, Niyogi P. Laplacian Eigenmaps for Dimensionality Reduction and Data Representation. Neural Computation, 2003, 15(6): 1373-1396 [7] Bian Zhaoqi, Zhang Xuegong. Pattern Recognition. Beijing, China: Tsinghua University Press, 2000 (in Chinese) (边肇祺,张学工.模式识别.北京:清华大学出版社, 2000) [8] Nene S A, Nayar S K, Murase H. Columbia Object Image Library (COIL20). Technical Report, CUCS-005-96. New York, USA: Columbia University. Department of Computer Science, 1996 [9] Asuncion A, Newman D J. UCI Machine Learning Repository [DB/OL]. [2007-04-13]. http://archive.ics.vci.edu [10] Duda R O, Hart P E, Stork D G. Pattern Classification. 2nd Edition. New York, USA: John Wiley & Sons, 2001 [11] Weinberger K Q, Saul L K. Unsupervised Learning of Image Manifolds by Semidefinite Programming. International Journal of Computer Vision, 2006, 70(1): 77-90
2009, Vol. 22 
