A Fast Data-Oriented Algorithm for Principal Component Analysis
YU Ying1, WANG Bin1,2, ZHANG Li-Ming1
1.Department of Electronics Engineering, Fudan University, Shanghai 200433 2.The Key Laboratory of Wave Scattering and Remote Sensing Information Ministry of Education, School of Information Science and Engineering, Fudan University, Shanghai 200433
Abstract Principal components analysis (PCA) for high-dimensional data is a difficult problem because the computational time and the space complexity rapidly increase as the data dimensions increase. A data-oriented and covariance-free PCA algorithm is proposed, inspired by the idea that the updated eigenvector in iteration is the weighted average of all samples. In a stationary environment or the condition that all training samples are available, the proposed algorithm is capable of overcoming the shortage of the conventional batch or incremental approaches. Furthermore, the convergence of the proposed algorithm is proved mathematically. Experimental results show that the most accurate solution is converged in a few iterations by the proposed algorithm.
[1] Haykin S. Neural Networks: A Comprehensive Foundation. 2nd Edition. Upper Saddle River, USA: Prentice Hall, 2001 [2] Golub G H, van Loan C F. Matrix Computation. 3rd Edition. Baltimore, USA: John Hopkins University Press, 1996 [3] Torralba A, Oliva A. Statistics of Natural Image Categories. Network: Computation in Neural Systems, 2003, 14(3): 391-412 [4] Oja E. A Simplified Neuron Model as a Principal Component Analyzer. Journal of Mathematical Biology, 1982, 15(3): 267-273 [5] Oja E, Karhunen J. On Stochastic Approximation of the Eigenvectors and Eigenvalues of the Expectation of a Random Matrix. Journal of Mathematical Analysis and Applications, 1985, 106: 69-84 [6] Sanger T D. Optimal Unsupervised Learning in a Single-Layer Linear Feedforward Neural Network. IEEE Trans on Neural Networks, 1989, 2(6): 459-473 [7] Weng Juyang, Zhang Yilu, Hwang W S. Candid Covariance-Free Incremental Principal Component Analysis. IEEE Trans on Pattern Analysis and Machine Intelligence, 2003, 25(8): 1034-1040
2009, Vol. 22 
