基于相对分类信息熵的进化特征选择算法<sup>*</sup>

doi:10.16451/j.cnki.issn1003-6059.201608002

Abstract
Figure/Table
References (0)
Related Citation (15)

Download: PDF (549 KB) HTML (1 KB)
Export: BibTeX | EndNote (RIS)

Abstract

Aiming at the problem of feature selection from datasets with discrete values, a feature selection approach via evolutionary computation based on relative classification information entropy is proposed. Genetic algorithm is used to search the optimal feature subset and the relative classification information entropy is employed to measure the significance of the feature subset. Specifically, the relative classification information entropy is used as fitness function, the solutions of the problems are encoded with binary number, and the next generation of individuals is produced by using roulette wheel method. The experimental results show that the proposed approach outperforms other methods in testing accuracy. Furthermore, the proposed approach is theoretically proved to be feasible.

Key words： Feature Selection Data Preprocessing Evolutionary Computation Genetic Algorithm Information Entropy

Received: 05 January 2016

ZTFLH:

TP 181

Fund:

Supported by National Natural Science Foundation of China (No.71371063), Natural Science Foundation of Hebei Province (No.F2013201220), Key Scientific Research Foundation of Education Department of Hebei Province (No.ZD20131028)

About author:: (ZHAI Junhai(Corresponding author), born in 1964, Ph.D., professor. His research interests include machine learning and data mining.)(LIU Bo, born in 1989, master student. His research interests include data mining.)(ZHANG Sufang, born in 1966, master, associate professor. Her research interests include machine learning.)

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	ZHAI Junhai
	LIU Bo
	ZHANG Sufang

Cite this article:

ZHAI Junhai,LIU Bo,ZHANG Sufang. Feature Selection via Evolutionary Computation Based on Relative Classification Information Entropy[J]. , 2016, 29(8): 682-690.

URL:

http://manu46.magtech.com.cn/Jweb_prai/EN/10.16451/j.cnki.issn1003-6059.201608002 OR http://manu46.magtech.com.cn/Jweb_prai/EN/Y2016/V29/I8/682

[1] GUYON I, ELISSEEFF A. An Introduction to Variable and Feature Selection. Journal of Machine Learning Research, 2003, 3: 1157-1182.
[2] BLUM A L, LANGLEY P. Selection of Relevant Features and Examples in Machine Learning. Artificial Intelligence, 1997, 97(1/2): 245-271.
[3] GUYON I, GUNN S R, NIKRAVESH M, et al. Feature Extraction: Foundations and Applications. Berlin, Germany: Springer, 2006.
[4] LIU H, YU L. Toward Integrating Feature Selection Algorithms for Classification and Clustering. IEEE Trans on Knowledge and Data Engineering, 2005, 17(4): 491-502.
[5] ALMUALLIM H, DIETTERICH T G. Learning Boolean Concepts in the Presence of Many Irrelevant Features. Artificial Intelligence, 1994, 69(1/2): 279-305.
[6] DASH M, LIU H. Consistency-Based Search in Feature Selection. Artificial Intelligence, 2003, 151(1/2): 155-176.
[7] YU L, LIU H. Efficient Feature Selection via Analysis of Relevance and Redundancy. Journal of Machine Learning Research, 2004, 5:1205-1224.
[8] BATTITI R. Using Mutual Information for Selecting Features in Supervised Neural Net Learning. IEEE Trans on Neural Networks, 1994, 5(4): 537-550.
[9] KWAK N, CHOI C H. Input Feature Selection by Mutual Information Based on Parzen Window. IEEE Trans on Pattern Analysis and Machine Intelligence, 2002, 24(12): 1667-1671.
[10] PENG H C, LONG F H, DING C. Feature Selection Based on Mutual Information: Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy. IEEE Trans on Pattern Analysis and Machine Intelligence, 2005, 27(8): 1226-1238.
[11] KOHAVI R, JOHN G H. Wrappers for Feature Subset Selection. Artificial Intelligence, 1997, 97(1/2): 273-324.
[12] SINDHWANI V, RAKSHIT S, DEODHARE D, et al. Feature Selection in MLPs and SVMs Based on Maximum Output Information. IEEE Trans on Neural Networks, 2004, 15(4): 937-948.
[13] YANG J B, SHEN K Q, ONG C J, et al. Feature Selection for MLP Neural Network: The Use of Random Permutation of Probabilistic Outputs. IEEE Trans on Neural Networks, 2009, 20(12): 1911-1922.
[14] QUINLAN J R. Induction of Decision Trees. Machine Learning, 1986, 1(1): 81-106.
[15] SHEN K Q, ONG C J, LI X P, et al. Feature Selection via Sensitivity Analysis of SVM Probabilistic Outputs. Machine Learning, 2008, 70(1): 1-20.
[16] PERKINS S, LACKER K, THEILER J. Grafting: Fast, Incremental Feature Selection by Gradient Descent in Function Space. Journal of Machine Learning Research, 2003, 3: 1333-1356.
[17] Holland J H. Adaptation in Natural and Artificial Systems. Cambridge, USA: MIT Press, 1992.
[18] YANG J, HONAVAR V G. Feature Subset Selection Using a Genetic Algorithm. IEEE Intelligent Systems, 1998, 13(2): 44-49.
[19] STEFANO C D, FONTANELLA F, MARROCCO C, et al. A GA-Based Feature Selection Approach with an Application to Handwritten Character Recognition. Pattern Recognition Letters, 2014, 35(1): 130-141.
[20] KABIR M M, SHAHJAHAN M, MURASE K. A New Local Search Based Hybrid Genetic Algorithm for Feature Selection. Neurocomputing, 2011, 74(17): 2914-2928.
[21] RAYMER M L, PUNCH W F, GOODMAN E D, et al. Dimensionality Reduction Using Genetic Algorithms. IEEE Trans on Evolutionary Computation, 2000, 4(2): 164-171.
[22] TSAI C F, EBERLE W, CHU C Y. Genetic Algorithms in Feature and Instance Selection. Knowledge-Based Systems, 2013, 39: 240-247.
[23] CHEN Z Y, LIN W C, KE S W, et al. Evolutionary Feature and Instance Selection for Traffic Sign Recognition. Computers in Industry, 2015, 74: 201-211.
[24] 苗夺谦,李道国.粗糙集理论、算法与应用.北京:清华大学出版社, 2008.
(MIAO D Q, LI D G. Rough Set Theories, Algorithms and Applications. Beijing, China: Tsinghua University Press, 2008.)
[25] 王宇平.进化计算的理论和方法.北京:科学出版社, 2011.
(WANG Y P. Evolutionary Computation Theory and Method. Beijing, China: Science Press, 2011.)
[26] 周涛,陆惠玲,张艳宁,等.基于Rough Set的高维特征选择混合遗传算法研究.南京大学学报(自然科学), 2015, 51(4):880-893.
(ZHOU T, LU H L, ZHANG Y N, et al. A New Hybrid Genetic Algorithm for High Dimension Feature Selection Based on Rough Set. Journal of Nanjing University (Nature Sciences), 2015, 51(4): 880-893.)
[27] 黄炜,黄志华.一种基于遗传算法和SVM的特征选择.计算机技术与发展, 2010, 20(6): 21-24.
(HUANG W, HUANG Z H. Feature Selection Based on Genetic Algorithm and SVM. Computer Technology and Development, 2010, 20(6): 21-24.)