Based on Locality Regularized Generalization Error Bound
XUE Hui1,2, CHEN Song-Can2
1.School of Computer Science Engineering,Southeast University,Nanjing 210096 2.College of Computer Science Technology,Nanjing University of Aeronautics and Astronautics,Nanjing 210016
Abstract：Feature selection is a hot topic in current pattern recognition. Filter and wrapper approaches are two of the most important policies to evaluate feature subsets in feature selection algorithms. However, they both can not guarantee the generalization performance of the following designed classifier. To solve these problems in the two approaches, a locality regularized generalization error bound is firstly introduced which embeds the manifold structure information hidden in the input samples. Furthermore, a hybrid filter-wrapper feature selection algorithm is proposed, which uses the locality regularized generalization error bound as the evaluation function as well as the locality regularization method as the classifier. As a result, the proposed algorithm can not only keep high computational efficiency, but also guarantee the good generalization performance of the following classifier. Experimental results validate the superiority of the algorithm.
[1] Duda R O,Hart P E,Stork D G.Pattern Classification.New York,USA: John Wiley Sons,2003 [2] Guyon I,Elisseeff A.An Introduction to Variable and Feature Selection.Journal of Machine Learning Research,2003,3: 1157-1182 [3] Zhan Dechuan,Zhou Zhihua.A Correlation Projection Score-Based Feature Selection Algorithm.Journal of Frontiers of Computer Science and Technology,2007,1(2): 138-145 (in Chinese) (詹德川,周志华.基于相关投影分的特征选择算法.计算机科学与探索,2007,1(2): 138-145) [4] Li Renpu,Wang Zhengou.A Structure-Adaptive Approach for Neural-Network-Based Feature Selection.Journal of Computer Research and Development,2002,39(12): 1613-1617 (in Chinese) (李仁璞,王正欧.一种结构自适应的神经网络特征选择方法.计算机研究与发展,2002,39(12): 1613-1617) [5] Ng W W Y,Yeung D S,Firth M,et al.Feature Selection Using Localized Generalization Error for Supervised Classification Problems Using RBFNN.Pattern Recognition,2008,41(12): 3706-3719 [6] Yeung D S,Ng W W Y,Wang D,et al.Localized Generalization Error and Its Application to Architecture Selection for Radial Basis Function Neural Network.IEEE Trans on Neural Networks,2007,18(5): 1294-1305 [7] Xue Hui,Chen Songcan,Zeng Xiaoqin.Classifier Learning with a New Locality Regularization Method.Pattern Recognition,2008,41(5): 1479-1490 [8] Belkin M,Niyogi P.Laplacian Eigenmaps and Spectral Technique for Embedding and Clustering // Proc of the 15th Annual Conference on Neural Information Processing Systems.Vancouver,Canada,2001: 585-591 [9] Mao Yong,Zhou Xiaobo,Xia Zheng,et al.A Survey for Study of Feature Selection Algorithms.Pattern Recognition and Artificial Intelligence,2007,20(2): 211-218 (in Chinese) (毛 勇,周晓波,夏 铮,等.特征选择算法研究综述.模式识别与人工智能,2007,20(2): 211-218) [10] Bhattacharjee A,Richards W G,Staunton J,et al.Classification of Humanlung Carcinomas by mRNA Expression Profiling Reveals Distinct Adenocarcinoma Subclasses // Proc of the National Academic Science,2001,98(24): 13790-13795 [11] Mitra P,Murthy C A,Pal S K.Unsupervised Feature Selection Using Feature Similarity.IEEE Trans on Pattern Analysis and Machine Intelligence,2002,24(3): 301-312
2011, Vol. 24 
