Abstract Most graph-based unsupervised feature selection methods choose l2,1-norm sparse regularization of the projection matrix instead of non-convex l2,0-norm constraint. However, the l2,1-norm regularization method selects features one by one according to the scores, without considering the correlation of features. Therefore, an unsupervised group feature selection method for graph optimization based on l2,0-norm sparsity and fuzzy similarity is proposed, and it simultaneously performs graph learning and feature selection. In graph learning, the similarity matrix with exact connected components is learned. In the process of feature selection, the number of non-zero rows of projection matrix is constrained to realize group feature selection. To solve the non-convex l2,0-norm constraint, the feature selection vector with elements of 0 or 1 is introduced to transform the l2,0-norm constraint problem into 0-1 integer programming problem, and the discrete 0-1 integer constraint is transformed into two continuous constraints to solve the problem. Finally, fuzzy similarity factor is introduced to extend the method and learn more accurate graph structure. Experiments on real datasets show the effectiveness of the proposed method.
Corresponding Authors:
ZHOU Shuisheng, Ph.D., professor. His research interests include optimization theory and algorithm, pattern recognition and application, intelligent information processing and machine learning.
About author:: MENG Tiantian, master student. Her research interests include machine learning and optimization theory and algorithm.TIAN Xinrun, master student. Her research interests include optimization theory and algorithm, and pattern recognition and application.
MENG Tiantian,ZHOU Shuisheng,TIAN Xinrun. Unsupervised Group Feature Selection Method for Graph Optimization Based on l2,0-norm Sparsity and Fuzzy Similarity[J]. Pattern Recognition and Artificial Intelligence, 2023, 36(1): 34-48.
[1] YANG M S, BENJAMIN J B M. Feature-Weighted Possibilistic C-means Clustering with a Feature-Reduction Framework. IEEE Transactions on Fuzzy Systems, 2021, 29(5): 1093-1106. [2] PARISOTTO S, CALATRONI L, BUGEAU A, et al. Variational Osmosis for Non-linear Image Fusion. IEEE Transactions on Image Processing, 2020, 29: 5507-5516. [3] ZHANG X Y, LIU C L, SUEN C Y.Towards Robust Pattern Re-cognition: A Review. Proceedings of the IEEE, 2020, 108(6): 894-922. [4] 孙林,黄金旭,徐久成,等.基于自适应鲸鱼优化算法和容错邻域粗糙集的特征选择算法.模式识别与人工智能, 2022, 35(2): 150-165. (SUN L, HUANG J X, XU J C.Feature Selection Based on Adaptive Whale Optimization Algorithm and Fault-Tolerance Neighborhood Rough Sets. Pattern Recognition and Artificial Intelligence, 2022, 35(2): 150-165.) [5] NIE F P, YANG S, ZHANG R, et al. A General Framework for Auto-Weighted Feature Selection via Global Redundancy Minimization. IEEE Transactions on Image Processing, 2019, 28(5): 2428-2438. [6] DENG W Y, LENDASSE A, ONG Y S, et al. Domain Adaption via Feature Selection on Explicit Feature Map. IEEE Transactions on Neural Networks and Learning Systems, 2019, 30(4): 1180-1190. [7] ZHAN S H, WU J G, HAN N, et al. Unsupervised Feature Extraction by Low-Rank and Sparsity Preserving Embedding. Neural Networks, 2019, 109: 56-66. [8] LIN W H, HUANG J F, SUEN C Y, et al. A Feature Extraction Model Based on Discriminative Graph Signals. Expert Systems with Applications, 2019, 139. DOI: 10.1016/j.eswa.2019.112861. [9] 孙世明,邓安生.基于最大决策边界的局部在线流特征选择.模式识别与人工智能, 2021, 34(12): 1131-1142. (SUN S M, DENG A S.Local Online Streaming Feature Selection Based on Max-Decision Boundary. Pattern Recognition and Artificial Intelligence, 2021, 34(12): 1131-1142.) [10] ZHOU P, LI P P, ZHAO S, et al. Feature Interaction for Streaming Feature Selection. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(10): 4691-4702. [11] FAN M Y, ZHANG X Q, HU J, et al. Adaptive Data Structure Regularized Multiclass Discriminative Feature Selection. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(10): 5859-5872. [12] YUAN A H, YOU M B, HE D J, et al. Convex Non-negative Matrix Factorization with Adaptive Graph for Unsupervised Feature Selection. IEEE Transactions on Cybernetics, 2022, 52(6): 5522-5534. [13] CAPÓ M, PÉREZ A, LOZANO J A, et al. A Cheap Feature Selection Approach for the K-means Algorithm. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(5): 2195-2208. [14] HE X F, CAI D, NIYOGI P.Laplacian Score for Feature Selection // Proc of the 18th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2005: 507-514. [15] WANG A G, AN N, YANG J, et al. Wrapper-Based Gene Selection with Markov Blanket. Computers in Biology and Medicine, 2017, 81: 11-23. [16] CHEN X J, YUAN G W, WANG W T, et al. Local Adaptive Projection Framework for Feature Selection of Labeled and Unlabeled Data. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(12): 6362-6373. [17] TAO H, HOU C P, NIE F P, et al. Effective Discriminative Feature Selection with Nontrivial Solution. IEEE Transactions on Neural Networks and Learning Systems, 2016, 27(4): 796-808. [18] CAI D, ZHANG C Y, HE X F.Unsupervised Feature Selection for Multi-cluster Data // Proc of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2010: 333-342. [19] YANG Y, SHEN H T, MA Z G, et al. l2,1-norm Regularized Discriminative Feature Selection for Unsupervised Learning // Proc of the 22nd International Joint Conference on Artificial Intelligence. San Francisco, USA: IJCAI, 2011, II: 1589-1594. [20] SHANG R H, WANG W B, STOLKIN R, et al. Subspace Lear-ning-Based Graph Regularized Feature Selection. Knowledge-Based Systems, 2016, 112: 152-165. [21] NIE F P, ZHU W, LI X L.Structured Graph Optimization for Unsupervised Feature Selection. IEEE Transactions on Knowledge and Data Engineering, 2021, 33(3): 1210-1222. [22] LI X L, ZHANG H, ZHANG R, et al. Generalized Uncorrelated Regression with Adaptive Graph for Unsupervised Feature Selection. IEEE Transactions on Neural Networks and Learning Systems, 2019, 30(5): 1587-1595. [23] ZHANG R, ZHANG Y X, LI X L, et al. Unsupervised Feature Selection via Adaptive Graph Learning and Constraint. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(3): 1355-1362. [24] CAI X, NIE F P, HUANG H.Exact Top-k Feature Selection via l2,0-norm Constraint // Proc of the 23rd International Joint Confe-rence on Artificial Intelligence. San Francisco, USA: IJCAI, 2013: 1240-1246. [25] DU X Z, NIE F P, WANG W Q, et al. Exploiting Combination Effect for Unsupervised Feature Selection by l2,0-norm. IEEE Transactions on Neural Networks and Learning Systems, 2019, 30(1): 201-214. [26] NIE F P, DONG X, TIAN L, et al. Unsupervised Feature Selection with Constrained l2,0-norm and Optimized Graph. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(4): 1702-1713. [27] PANG T J, NIE F P, HAN J W, et al. Efficient Feature Selection via l2,0-norm Constrained Sparse Regression. IEEE Transactions on Knowledge and Data Engineering, 2019, 31(5): 880-893. [28] NIE F P, WANG X Q, HENG H.Clustering and Projected Clustering with Adaptive Neighbors // Proc of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2014: 977-986. [29] FAN K.On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations. Proceedings of the National Academy of Sciences of the United States of America, 1949, 35(11): 652-655. [30] WU B Y, GHANEM B.lp-Box ADMM: A Versatile Framework for Integer Programming. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(7): 1695-1708. [31] HUANG J, NIE F P, HUANG H.A New Simplex Sparse Learning Model to Measure Data Similarity for Clustering // Proc of the 24th International Conference on Artificial Intelligence. San Francisco, USA: IJCAI, 2015: 3569-3575.
2023, Vol. 36 
