|
|
|
| A Deep Takagi-Sugeno-Kang Fuzzy Classifier for Imbalanced Data |
| BIAN Zekang1, ZHANG Jin1, WANG Shitong1 |
| 1. School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122 |
|
|
|
|
Abstract Inspired by ensemble learning, a deep Takagi-Sugeno-Kang fuzzy classifier for imbalanced data(ID-TSK-FC) is proposed to enhance the generalization capability and maintain good linguistic interpretability of TSK fuzzy classifier on imbalanced data. ID-TSK-FC is composed of an imbalanced global linear regression sub-classifier(IGLRc) and several imbalanced TSK fuzzy sub-classifiers(I-TSK-FCs). According to the human cognitive behavior "from wholly coarse to locally fine" and the stacked generalization principle, ID-TSK-FC firstly trains an IGLRc on all training samples to obtain a wholly coarse classification result. Then, the nonlinear training samples in the original training samples are classified according to the output of IGLRc. Next, several I-TSK-FCs are generated using a stacked depth structure on the nonlinear training samples to achieve a locally fine result. Finally, the minimum distance voting principle is applied on the outputs of stacked IGLRc and all I-TSK-FCs to obtain the final output of ID-TSK-FC. The experimental results confirm that ID-TSK-FC not only holds interpretability based on feature importance, but also holds at least comparable generalization capability and linguistic interpretability.
|
|
Received: 29 November 2022
|
|
|
| Fund:National Key Research and Development Program(No.2022YFE0112400), Natural Science Foundation of Jiangsu Province(No.BK20191331), Natural Science Key Research Project of Jiangsu Provincial Department of Education(No.22KJA520009) |
|
Corresponding Authors:
WANG Shitong, master, professor. His research interests include artificial intelligence and pattern recognition.
|
| About author:: BIAN Zekang, Ph.D., lecturer. His research interests include artificial intelligence and pattern recognitionZHANG Jin, Ph.D. candidate. His research interests include artificial intelligence and pattern recognition. |
|
|
|
[1] 阳爱民,胡运发.一种核模糊分类器的规则生成方法.模式识别与人工智能, 2006, 19(2): 196-202.
(YANG A M, HU Y F. A Method of Generating Rules with a Kernel Fuzzy Classifier. Pattern Recognition and Artificial Intelligence, 2006, 19(2): 196-202.)
[2] 周塔,邓赵红,蒋亦樟,等.基于训练空间重构的多模块TSK模糊系统.软件学报, 2020, 31(11): 3506-3518.
(ZHOU T, DENG Z H, JIANG Y Z, et al. Multi-module TSK Fuzzy System Based on Training Space Recognition. Journal of Software, 2020, 31(11): 3506-3518.)
[3] RAMENTOL E, VLUYMANS S, VERBIEST N, et al. IFROWANN: Imbalanced Fuzzy-Rough Ordered Weighted Average Nearest Neighbor Classification. IEEE Transactions on Fuzzy Systems, 2015, 23(5): 1622-1637.
[4] ZHANG X Y, WEI X, LI F L, et al. Fuzzy Support Vector Machine with Imbalanced Regulator and Its Application in Stroke Classification // Proc of the IEEE 5th International Conference on Big Data Computing Service and Applications. Washington, USA: IEEE, 2019: 290-295.
[5] XU L, CHOW M Y, TAYLOR L S. Data Mining Based Fuzzy Cla-ssification Algorithm for Imbalanced Data // Proc of the IEEE International Conference on Fuzzy Systems. Washington, USA: IEEE, 2006: 825-830.
[6] VILLAR P, FERNÁNDEZ A, HERRERA F. Studying the Behavior of a Multiobjective Genetic Algorithm to Design Fuzzy Rule-Based Classification Systems for Imbalanced Data-Sets // Proc of the IEEE International Conference on Fuzzy Systems. Washington, USA: IEEE, 2011: 1239-1246.
[7] MAHDIZADEH M, EFTEKHARI M. Designing Fuzzy Imbalanced Classifier Based on the Subtractive Clustering and Genetic Progra-mming // Proc of the 13th Iranian Conference on Fuzzy Systems. Washington, USA: IEEE, 2013. DOI: 10.1109/IFSC.2013.6675611.
[8] LESKI J M, CZABAŃSKI R, JEZEWSKI M, et al. Fuzzy Ordered C-means Clustering and Least Angle Regression for Fuzzy Rule-Based Classifier: Study for Imbalanced Data. IEEE Transactions on Fuzzy Systems, 2020, 28(11): 2799-2813.
[9] SANZ J A, BERNARDO D, HERRERA F, et al. A Compact Evolutionary Interval-Valued Fuzzy Rule-Based Classification System for the Modeling and Prediction of Real-World Financial Applications with Imbalanced Data. IEEE Transactions on Fuzzy Systems, 2015, 23(4): 973-990.
[10] GU X Q, CHUNG F L, ISHIBUCHI H, et al. Imbalanced TSK Fuzzy Classifier by Cross-Class Bayesian Fuzzy Clustering and Imbalance Learning. IEEE Transactions on Systems, Man, and Cybernetics(Systems), 2017, 47(8): 2005-2020.
[11] DUAN S Y, YU S J, PRÍNCIPE J C. Modularizing Deep Learning via Pairwise Learning with Kernels. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(4): 1441-1451.
[12] ZHU T Y, LI K Z, HERRERO P, et al. Deep Learning for Diabetes: A Systematic Review. IEEE Journal of Biomedical and Health Informatics, 2021, 25(7): 2744-2757.
[13] MAHMUD M, KAISER M S, HUSSAIN A, et al. Applications of Deep Learning and Reinforcement Learning to Biological Data. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(6): 2063-2079.
[14] WANG G J, ZHOU T, CHOI K S, et al. A Deep-Ensemble-Level-Based Interpretable Takagi-Sugeno-Kang Fuzzy Classifier for Imba-lanced Data. IEEE Transactions on Cybernetics, 2022, 52(5): 3805-3818.
[15] SIGLETOS G, PALIOURAS G, SPYROPOULOS C D, et al. Combining Information Extraction Systems Using Voting and Stacked Generalization. Journal of Machine Learning Research, 2005, 6: 1751-1782.
[16] WANG S T, CHUNG F L, WU J, et al. Least Learning Machine and Its Experimental Studies on Regression Capability. Applied Soft Computing, 2014, 21: 677-684.
[17] WANG S T, JIANG Y Z, CHUNG F L, et al. Feedforward Kernel Neural Networks, Generalized Least Learning Machine, and Its Deep Learning with Application to Image Classification. Applied Soft Computing, 2015, 37: 125-141.
[18] QIN B, CHUNG F L, WANG S T. Biologically Plausible Fuzzy-Knowledge-Out and Its Induced Wide Learning of Interpretable TSK Fuzzy Classifiers. IEEE Transactions on Fuzzy Systems, 2020, 28(7): 1276-1290.
[19] GU S H, CHUNG F L, WANG S T. A Novel Deep Fuzzy Classifier by Stacking Adversarial Interpretable TSK Fuzzy Sub-Classifiers with Smooth Gradient Information. IEEE Transactions on Fuzzy Systems, 2020, 28(7): 1369-1382.
[20] ZHANG Y P, ISHIBUCHI H, WANG S T. Deep Takagi-Sugeno-Kang Fuzzy Classifier with Shared Linguistic Fuzzy Rules. IEEE Transactions on Fuzzy Systems, 2018, 26(3): 1535-1549.
[21] QIN B, CHUNG F L, WANG S T. KAT: A Knowledge Adversa-rial Training Method for Zero-Order Takagi-Sugeno-Kang Fuzzy Classifiers. IEEE Transactions on Cybernetics, 2022, 52(7): 6857-6871.
[22] LICHTENBERG J M, ŞIMŞEK Ö. Simple Regression Models. Proceedings of Machine Learning Research, 2017, 58: 13-25.
[23] KÖNIG G, MOLNAR C, BISCHL B, et al. Relative Feature Importance // Proc of the 25th International Conference on Pattern Recognition. Washington, USA: IEEE, 2020: 9318-9325.
[24] YE H S, LUO L, ZHANG Z H. Accelerated Proximal Subsampled Newton Method. IEEE Transactions on Neural Networks and Lear-ning Systems, 2020, 32(10): 4374-4388.
[25] ROOSTA-KHORASANI F, MAHONEY M W. Sub-Sampled Newton Methods. Mathematical Programming, 2019, 174(1): 293-326.
[26] KIM D. Accelerated Proximal Point Method for Maximally Mono-tone Operators. Mathematical Programming, 2021, 190(1/2): 57-87.
[27] ZHOU T, CHUNG F L, WANG S T. Deep TSK Fuzzy Classifier with Stacked Generalization and Triplely Concise Interpretability Guarantee for Large Data. IEEE Transactions on Fuzzy Systems, 2017, 25(5): 1207-1221.
[28] GACTO M J, ALCALÁ R, HERRERA F. Interpretability of Lin-guistic Fuzzy Rule-Based Systems: An Overview of Interpretability Measures. Information Sciences, 2011, 181(20): 4340-4360.
[29] DEMŠAR J. Statistical Comparisons of Classifiers over Multiple Data Sets. Journal of Machine Learning Research, 2006, 7: 1-30. |
|
|
|
2023, Vol. 36 
