基于相对分类信息熵的进化特征选择算法<sup>*</sup>

doi:10.16451/j.cnki.issn1003-6059.201608002

摘要
图/表
参考文献(0)
相关文章 (15)

全文: PDF (549 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS) 背景资料

摘要

针对离散值数据集特征选择问题，提出基于相对分类信息熵的进化特征选择算法.使用遗传算法搜索最优特征子集，使用相对分类信息熵度量特征子集的重要性.以相对分类信息熵作为适应度函数，使用二进制编码问题的解，使用赌轮方法选择产生下一代个体.实验表明文中算法在测试精度上优于其它方法，此外还从理论上证明文中算法的可行性.

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	翟俊海
	刘博
	张素芳

Abstract：

Aiming at the problem of feature selection from datasets with discrete values, a feature selection approach via evolutionary computation based on relative classification information entropy is proposed. Genetic algorithm is used to search the optimal feature subset and the relative classification information entropy is employed to measure the significance of the feature subset. Specifically, the relative classification information entropy is used as fitness function, the solutions of the problems are encoded with binary number, and the next generation of individuals is produced by using roulette wheel method. The experimental results show that the proposed approach outperforms other methods in testing accuracy. Furthermore, the proposed approach is theoretically proved to be feasible.

Key words： Feature Selection Data Preprocessing Evolutionary Computation Genetic Algorithm Information Entropy

收稿日期: 2016-01-05

ZTFLH:

TP 181

基金资助:

国家自然科学基金项目(No.71371063)、河北省自然科学基金项目(No.F2013201220)、河北省高等学校科学技术研究重点项目(No.ZD20131028)资助

作者简介: 翟俊海(通讯作者),男,1964年生,博士,教授,主要研究方向为机器学习、数据挖掘.E-mail:mczjh@126.com.刘博,男,1989年生,硕士研究生,主要研究方向为数据挖掘.E-mail:494279884@qq.com.张素芳,女,1966年生,硕士,副教授,主要研究方向为机器学习.E-mail:mczsf@126.com.

引用本文:

翟俊海，刘博，张素芳. 基于相对分类信息熵的进化特征选择算法^*[J]. 模式识别与人工智能, 2016, 29(8): 682-690. ZHAI Junhai, LIU Bo, ZHANG Sufang. Feature Selection via Evolutionary Computation Based on Relative Classification Information Entropy. , 2016, 29(8): 682-690.

链接本文:

http://manu46.magtech.com.cn/Jweb_prai/CN/10.16451/j.cnki.issn1003-6059.201608002 或 http://manu46.magtech.com.cn/Jweb_prai/CN/Y2016/V29/I8/682

[1] GUYON I, ELISSEEFF A. An Introduction to Variable and Feature Selection. Journal of Machine Learning Research, 2003, 3: 1157-1182.
[2] BLUM A L, LANGLEY P. Selection of Relevant Features and Examples in Machine Learning. Artificial Intelligence, 1997, 97(1/2): 245-271.
[3] GUYON I, GUNN S R, NIKRAVESH M, et al. Feature Extraction: Foundations and Applications. Berlin, Germany: Springer, 2006.
[4] LIU H, YU L. Toward Integrating Feature Selection Algorithms for Classification and Clustering. IEEE Trans on Knowledge and Data Engineering, 2005, 17(4): 491-502.
[5] ALMUALLIM H, DIETTERICH T G. Learning Boolean Concepts in the Presence of Many Irrelevant Features. Artificial Intelligence, 1994, 69(1/2): 279-305.
[6] DASH M, LIU H. Consistency-Based Search in Feature Selection. Artificial Intelligence, 2003, 151(1/2): 155-176.
[7] YU L, LIU H. Efficient Feature Selection via Analysis of Relevance and Redundancy. Journal of Machine Learning Research, 2004, 5:1205-1224.
[8] BATTITI R. Using Mutual Information for Selecting Features in Supervised Neural Net Learning. IEEE Trans on Neural Networks, 1994, 5(4): 537-550.
[9] KWAK N, CHOI C H. Input Feature Selection by Mutual Information Based on Parzen Window. IEEE Trans on Pattern Analysis and Machine Intelligence, 2002, 24(12): 1667-1671.
[10] PENG H C, LONG F H, DING C. Feature Selection Based on Mutual Information: Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy. IEEE Trans on Pattern Analysis and Machine Intelligence, 2005, 27(8): 1226-1238.
[11] KOHAVI R, JOHN G H. Wrappers for Feature Subset Selection. Artificial Intelligence, 1997, 97(1/2): 273-324.
[12] SINDHWANI V, RAKSHIT S, DEODHARE D, et al. Feature Selection in MLPs and SVMs Based on Maximum Output Information. IEEE Trans on Neural Networks, 2004, 15(4): 937-948.
[13] YANG J B, SHEN K Q, ONG C J, et al. Feature Selection for MLP Neural Network: The Use of Random Permutation of Probabilistic Outputs. IEEE Trans on Neural Networks, 2009, 20(12): 1911-1922.
[14] QUINLAN J R. Induction of Decision Trees. Machine Learning, 1986, 1(1): 81-106.
[15] SHEN K Q, ONG C J, LI X P, et al. Feature Selection via Sensitivity Analysis of SVM Probabilistic Outputs. Machine Learning, 2008, 70(1): 1-20.
[16] PERKINS S, LACKER K, THEILER J. Grafting: Fast, Incremental Feature Selection by Gradient Descent in Function Space. Journal of Machine Learning Research, 2003, 3: 1333-1356.
[17] Holland J H. Adaptation in Natural and Artificial Systems. Cambridge, USA: MIT Press, 1992.
[18] YANG J, HONAVAR V G. Feature Subset Selection Using a Genetic Algorithm. IEEE Intelligent Systems, 1998, 13(2): 44-49.
[19] STEFANO C D, FONTANELLA F, MARROCCO C, et al. A GA-Based Feature Selection Approach with an Application to Handwritten Character Recognition. Pattern Recognition Letters, 2014, 35(1): 130-141.
[20] KABIR M M, SHAHJAHAN M, MURASE K. A New Local Search Based Hybrid Genetic Algorithm for Feature Selection. Neurocomputing, 2011, 74(17): 2914-2928.
[21] RAYMER M L, PUNCH W F, GOODMAN E D, et al. Dimensionality Reduction Using Genetic Algorithms. IEEE Trans on Evolutionary Computation, 2000, 4(2): 164-171.
[22] TSAI C F, EBERLE W, CHU C Y. Genetic Algorithms in Feature and Instance Selection. Knowledge-Based Systems, 2013, 39: 240-247.
[23] CHEN Z Y, LIN W C, KE S W, et al. Evolutionary Feature and Instance Selection for Traffic Sign Recognition. Computers in Industry, 2015, 74: 201-211.
[24] 苗夺谦,李道国.粗糙集理论、算法与应用.北京:清华大学出版社, 2008.
(MIAO D Q, LI D G. Rough Set Theories, Algorithms and Applications. Beijing, China: Tsinghua University Press, 2008.)
[25] 王宇平.进化计算的理论和方法.北京:科学出版社, 2011.
(WANG Y P. Evolutionary Computation Theory and Method. Beijing, China: Science Press, 2011.)
[26] 周涛,陆惠玲,张艳宁,等.基于Rough Set的高维特征选择混合遗传算法研究.南京大学学报(自然科学), 2015, 51(4):880-893.
(ZHOU T, LU H L, ZHANG Y N, et al. A New Hybrid Genetic Algorithm for High Dimension Feature Selection Based on Rough Set. Journal of Nanjing University (Nature Sciences), 2015, 51(4): 880-893.)
[27] 黄炜,黄志华.一种基于遗传算法和SVM的特征选择.计算机技术与发展, 2010, 20(6): 21-24.
(HUANG W, HUANG Z H. Feature Selection Based on Genetic Algorithm and SVM. Computer Technology and Development, 2010, 20(6): 21-24.)