Abstract Inspired by ensemble learning, a deep Takagi-Sugeno-Kang fuzzy classifier for imbalanced data(ID-TSK-FC) is proposed to enhance the generalization capability and maintain good linguistic interpretability of TSK fuzzy classifier on imbalanced data. ID-TSK-FC is composed of an imbalanced global linear regression sub-classifier(IGLRc) and several imbalanced TSK fuzzy sub-classifiers(I-TSK-FCs). According to the human cognitive behavior "from wholly coarse to locally fine" and the stacked generalization principle, ID-TSK-FC firstly trains an IGLRc on all training samples to obtain a wholly coarse classification result. Then, the nonlinear training samples in the original training samples are classified according to the output of IGLRc. Next, several I-TSK-FCs are generated using a stacked depth structure on the nonlinear training samples to achieve a locally fine result. Finally, the minimum distance voting principle is applied on the outputs of stacked IGLRc and all I-TSK-FCs to obtain the final output of ID-TSK-FC. The experimental results confirm that ID-TSK-FC not only holds interpretability based on feature importance, but also holds at least comparable generalization capability and linguistic interpretability.
Fund:National Key Research and Development Program(No.2022YFE0112400), Natural Science Foundation of Jiangsu Province(No.BK20191331), Natural Science Key Research Project of Jiangsu Provincial Department of Education(No.22KJA520009)
Corresponding Authors:WANG Shitong, master, professor. His research interests include artificial intelligence and pattern recognition.
About author:: BIAN Zekang, Ph.D., lecturer. His research interests include artificial intelligence and pattern recognitionZHANG Jin, Ph.D. candidate. His research interests include artificial intelligence and pattern recognition.
BIAN Zekang,ZHANG Jin,WANG Shitong. A Deep Takagi-Sugeno-Kang Fuzzy Classifier for Imbalanced Data[J]. Pattern Recognition and Artificial Intelligence, 2023, 36(3): 211-224.
[1] 阳爱民,胡运发.一种核模糊分类器的规则生成方法.模式识别与人工智能, 2006, 19(2): 196-202. (YANG A M, HU Y F. A Method of Generating Rules with a Kernel Fuzzy Classifier. Pattern Recognition and Artificial Intelligence, 2006, 19(2): 196-202.) [2] 周塔,邓赵红,蒋亦樟,等.基于训练空间重构的多模块TSK模糊系统.软件学报, 2020, 31(11): 3506-3518. (ZHOU T, DENG Z H, JIANG Y Z, et al. Multi-module TSK Fuzzy System Based on Training Space Recognition. Journal of Software, 2020, 31(11): 3506-3518.) [3] RAMENTOL E, VLUYMANS S, VERBIEST N, et al. IFROWANN: Imbalanced Fuzzy-Rough Ordered Weighted Average Nearest Neighbor Classification. IEEE Transactions on Fuzzy Systems, 2015, 23(5): 1622-1637. [4] ZHANG X Y, WEI X, LI F L, et al. Fuzzy Support Vector Machine with Imbalanced Regulator and Its Application in Stroke Classification // Proc of the IEEE 5th International Conference on Big Data Computing Service and Applications. Washington, USA: IEEE, 2019: 290-295. [5] XU L, CHOW M Y, TAYLOR L S. Data Mining Based Fuzzy Cla-ssification Algorithm for Imbalanced Data // Proc of the IEEE International Conference on Fuzzy Systems. Washington, USA: IEEE, 2006: 825-830. [6] VILLAR P, FERNÁNDEZ A, HERRERA F. Studying the Behavior of a Multiobjective Genetic Algorithm to Design Fuzzy Rule-Based Classification Systems for Imbalanced Data-Sets // Proc of the IEEE International Conference on Fuzzy Systems. Washington, USA: IEEE, 2011: 1239-1246. [7] MAHDIZADEH M, EFTEKHARI M. Designing Fuzzy Imbalanced Classifier Based on the Subtractive Clustering and Genetic Progra-mming // Proc of the 13th Iranian Conference on Fuzzy Systems. Washington, USA: IEEE, 2013. DOI: 10.1109/IFSC.2013.6675611. [8] LESKI J M, CZABAŃSKI R, JEZEWSKI M, et al. Fuzzy Ordered C-means Clustering and Least Angle Regression for Fuzzy Rule-Based Classifier: Study for Imbalanced Data. IEEE Transactions on Fuzzy Systems, 2020, 28(11): 2799-2813. [9] SANZ J A, BERNARDO D, HERRERA F, et al. A Compact Evolutionary Interval-Valued Fuzzy Rule-Based Classification System for the Modeling and Prediction of Real-World Financial Applications with Imbalanced Data. IEEE Transactions on Fuzzy Systems, 2015, 23(4): 973-990. [10] GU X Q, CHUNG F L, ISHIBUCHI H, et al. Imbalanced TSK Fuzzy Classifier by Cross-Class Bayesian Fuzzy Clustering and Imbalance Learning. IEEE Transactions on Systems, Man, and Cybernetics(Systems), 2017, 47(8): 2005-2020. [11] DUAN S Y, YU S J, PRÍNCIPE J C. Modularizing Deep Learning via Pairwise Learning with Kernels. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(4): 1441-1451. [12] ZHU T Y, LI K Z, HERRERO P, et al. Deep Learning for Diabetes: A Systematic Review. IEEE Journal of Biomedical and Health Informatics, 2021, 25(7): 2744-2757. [13] MAHMUD M, KAISER M S, HUSSAIN A, et al. Applications of Deep Learning and Reinforcement Learning to Biological Data. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(6): 2063-2079. [14] WANG G J, ZHOU T, CHOI K S, et al. A Deep-Ensemble-Level-Based Interpretable Takagi-Sugeno-Kang Fuzzy Classifier for Imba-lanced Data. IEEE Transactions on Cybernetics, 2022, 52(5): 3805-3818. [15] SIGLETOS G, PALIOURAS G, SPYROPOULOS C D, et al. Combining Information Extraction Systems Using Voting and Stacked Generalization. Journal of Machine Learning Research, 2005, 6: 1751-1782. [16] WANG S T, CHUNG F L, WU J, et al. Least Learning Machine and Its Experimental Studies on Regression Capability. Applied Soft Computing, 2014, 21: 677-684. [17] WANG S T, JIANG Y Z, CHUNG F L, et al. Feedforward Kernel Neural Networks, Generalized Least Learning Machine, and Its Deep Learning with Application to Image Classification. Applied Soft Computing, 2015, 37: 125-141. [18] QIN B, CHUNG F L, WANG S T. Biologically Plausible Fuzzy-Knowledge-Out and Its Induced Wide Learning of Interpretable TSK Fuzzy Classifiers. IEEE Transactions on Fuzzy Systems, 2020, 28(7): 1276-1290. [19] GU S H, CHUNG F L, WANG S T. A Novel Deep Fuzzy Classifier by Stacking Adversarial Interpretable TSK Fuzzy Sub-Classifiers with Smooth Gradient Information. IEEE Transactions on Fuzzy Systems, 2020, 28(7): 1369-1382. [20] ZHANG Y P, ISHIBUCHI H, WANG S T. Deep Takagi-Sugeno-Kang Fuzzy Classifier with Shared Linguistic Fuzzy Rules. IEEE Transactions on Fuzzy Systems, 2018, 26(3): 1535-1549. [21] QIN B, CHUNG F L, WANG S T. KAT: A Knowledge Adversa-rial Training Method for Zero-Order Takagi-Sugeno-Kang Fuzzy Classifiers. IEEE Transactions on Cybernetics, 2022, 52(7): 6857-6871. [22] LICHTENBERG J M, ŞIMŞEK Ö. Simple Regression Models. Proceedings of Machine Learning Research, 2017, 58: 13-25. [23] KÖNIG G, MOLNAR C, BISCHL B, et al. Relative Feature Importance // Proc of the 25th International Conference on Pattern Recognition. Washington, USA: IEEE, 2020: 9318-9325. [24] YE H S, LUO L, ZHANG Z H. Accelerated Proximal Subsampled Newton Method. IEEE Transactions on Neural Networks and Lear-ning Systems, 2020, 32(10): 4374-4388. [25] ROOSTA-KHORASANI F, MAHONEY M W. Sub-Sampled Newton Methods. Mathematical Programming, 2019, 174(1): 293-326. [26] KIM D. Accelerated Proximal Point Method for Maximally Mono-tone Operators. Mathematical Programming, 2021, 190(1/2): 57-87. [27] ZHOU T, CHUNG F L, WANG S T. Deep TSK Fuzzy Classifier with Stacked Generalization and Triplely Concise Interpretability Guarantee for Large Data. IEEE Transactions on Fuzzy Systems, 2017, 25(5): 1207-1221. [28] GACTO M J, ALCALÁ R, HERRERA F. Interpretability of Lin-guistic Fuzzy Rule-Based Systems: An Overview of Interpretability Measures. Information Sciences, 2011, 181(20): 4340-4360. [29] DEMŠAR J. Statistical Comparisons of Classifiers over Multiple Data Sets. Journal of Machine Learning Research, 2006, 7: 1-30.
2023, Vol. 36 
