Abstract The loss of the correctly classified samples is counted as zero by classical spherical classifier in extremely imbalanced classification. The decision function is constructed only by misclassified samples. In this paper, a smooth reward-penalty loss function with lower bound is proposed. The loss of the correctly classified samples is counted as negative in the proposed loss function. Therefore, the reward of the objective function can be realized and the interference of noise near the boundary can be avoided. Based on maximum margin of twin spheres support vector machine, a maximum margin of twin sphere model via combined reward-penalty loss function with lower bound(RPMMTS) is established. Two concentric spheres are constructed by RPMMTS using Newton's method. The majority samples are captured in the small sphere and the extra space are eliminated at the same time. By increasing the margin between two concentric spheres, the minority samples are pushed out of the large sphere as many as possible. Experimental results show that the proposed loss function makes RPMMTS better than other unbalanced classification algorithms in classification performance.
Fund:National Natural Science Foundation of China(No.61772020)
Corresponding Authors:
ZHOU Shuisheng, Ph.D., professor. His research interests include optimization theory and algorithm, pattern recognition and application, intelligent information processing and machine lear-ning.
About author:: KANG Qian, master student. Her research interests include optimization theory and algorithm, and pattern recognition and application.
KANG Qian,ZHOU Shuisheng. Maximum Margin of Twin Sphere Model via Combined Smooth Reward-Penalty Loss Function with Lower Bound[J]. , 2021, 34(10): 885-897.
[1] ARASHLOO S R, KITTLER J. Robust One-Class Kernel Spectral Regression. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(3): 999-1013. [2] 平 瑞,周水生,李 冬.高度不平衡数据的代价敏感随机森林分类算法.模式识别与人工智能, 2020, 33(3): 249-257. (PING R, ZHOU S S, LI D. Cost Sensitive Random Forest Classification Algorithm for Highly Unbalanced Data. Pattern Recognition and Artificial Intelligence, 2020, 33(3): 249-257.) [3] BELLINGER C, SHARMA S, JAPKOWICZ N. One-Class versus Binary Classification: Which and When // Proc of the 11th International Conference on Machine Learning and Applications. Washington, USA: IEEE, 2012: 102-106. [4] DAS V, PATHAK V, SHARMA S, et al. Network Intrusion Detection System Based on Machine Learning Algorithms. International Journal of Computer Science and Information Technology, 2010, 2(6): 138-151. [5] PLAKIAS S, BOUTALIS Y S. Exploiting the Generative Adversarial Framework for One-Class Multi-dimensional Fault Detection. Neurocomputing, 2019, 332: 396-405. [6] ABDALLAH A, MAAROF M A, ZAINAL A. Fraud Detection System: A Survey. Journal of Network and Computer Applications, 2016, 68: 90-113. [7] ROY R, GEORGE K T. Detecting Insurance Claims Fraud Using Machine Learning Techniques // Proc of the International Confe-rence on Circuit Power and Computing Technologies. Washington, USA: IEEE, 2017. DOI: 10.1109/ICCPCT.2017.8074258. [8] YU M, YU Y Z, RHUMA A, et al. An Online One Class Support Vector Machine-Based Person-Specific Fall Detection System for Monitoring an Elderly Individual in a Room Environment. IEEE Journal of Biomedical and Health Informatics, 2013, 17(6): 1002-1014. [9] SUN J Y, SHAO J, HE C K. Abnormal Event Detection for Video Surveillance Using Deep One-Class Learning. Multimedia Tools and Applications, 2019, 78: 3633-3647. [10] AMRAEE S, VAFAEI A, JAMSHIDI K, et al. Abnormal Event Detection in Crowded Scenes Using One-Class SVM. Signal, Image and Video Processing, 2018, 12(6): 1115-1123. [11] PAN T T, ZHAO J H, WU W, et al. Learning Imbalanced Datasets Based on SMOTE and Gaussian Distribution. Information Sciences, 2020, 512: 1214-1233. [12] TAX D M J, DUIN R P W. Support Vector Data Description. Machine Learning, 2004, 54: 45-66. [13] WU M R, YE J P. A Small Sphere and Large Margin Approach for Novelty Detection Using Training Data with Outliers. IEEE Tran-sactions on Pattern Analysis and Machine Intelligence, 2009, 31(11): 2088-2092. [14] XU Y X. Maximum Margin of Twin Spheres Support Vector Machine for Imbalanced Data Classification. IEEE Transactions on Cybernetics, 2017, 47(6): 1540-1550. [15] WANG Z G, ZHAO Z S, WENG S F, et al. Solving One-Class Problem with Outlier Examples by SVM. Neurocomputing, 2015, 149: 100-105. [16] LE T, TRAN D, MA W L, et al. An Optimal Sphere and Two Large Margins Approach for Novelty Detection // Proc of the International Joint Conference on Neural Networks. Washington, USA: IEEE, 2010. DOI: 10.1109/IJCNN.2010.5596456. [17] XU Y T, YANG Z J, ZHANG Y Q, et al. A Maximum Margin and Minimum Volume Hyper-Spheres Machine with Pinball Loss for Imbalanced Data Classification. Knowledge-Based Systems, 2016, 95: 75-85. [18] XU Y T, ZHANG Y Q, ZHAO J, et al. KNN-Based Maximum Margin and Minimum Volume Hyper-Sphere Machine for Imba-lanced Data Classification. International Journal of Machine Lear-ning and Cybernetics, 2019, 10: 357-368. [19] BREIMAN L. Hinging Hyperplanes for Regression, Classification, and Function Approximation. IEEE Transactions on Information Theory, 1993, 39(3): 999-1013. [20] HUANG X L, SHI L, SUYKENS J A K. Support Vector Machine Classifier with Pinball Loss. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(5): 984-997. [21] XU Y T, QIAN W, PANG X Y, et al. Maximum Margin of Twin Spheres Machine with Pinball Loss for Imbalanced Data Classification. Applied Intelligence, 2018, 48: 23-34. [22] 方宝富,马云婷,王在俊,等.稀疏奖励下基于情感的异构多智能体强化学习.模式识别与人工智能, 2021, 34(3): 223-231. (FANG B F, MA Y T, WANG Z J, et al. Emotion-Based Heterogeneous Multi-agent Reinforcement Learning with Sparse Reward. Pattern Recognition and Artificial Intelligence, 2021, 34(3): 223-231.) [23] ANAND P. A Combined Reward-Penalty Loss Function Based Su-pport Vector Machine // Proc of the International CET Conference on Control, Communication, and Computing. Washington, USA: IEEE, 2018: 352-355. [24] 顾苏杭,王士同.基于数据点本身及其位置关系辅助信息挖掘的分类方法.模式识别与人工智能, 2018, 31(3): 197-207. (GU S H, WANG S T. Classification Approach by Mining Betweenness Information beyond Data Points Themselves. Pattern Recognition and Artificial Intelligence, 2018, 31(3): 197-207.) [25] YI L. A Note on Margin-Based Loss Functions in Classification. Statistics and Probability Letters, 2004, 68(1): 73-82. [26] SCHLKÖPF B, HERBRICH R, SMOLA A J. A Generalized Representer Theorem // Proc of the International Conference on Computational Learning Theory. Berlin, Germany: Springer, 2001: 416-426. [27] 邓乃扬,田英杰.数据挖掘中的新方法:支持向量机.北京:科学出版社, 2004. (DENG N Y, TIAN Y J. A New Method in Data Mining: Support Vector Machine. Beijing, China: Science Press, 2004.) [28] IRANMEHR A, MASNADI-SHIRAZI H, VASCONCELOS N. Cost-Sensitive Support Vector Machines. Neurocomputing, 2019, 343: 50-64. [29] TAO X M, LI Q, REN C, et al. Affinity and Class Probability-Based Fuzzy Support Vector Machine for Imbalanced Data Sets. Neural Networks, 2020, 122: 289-307. [30] CHEN L, ZHOU S S. Sparse Algorithm for Robust LSSVM in Primal Space. Neurocomputing, 2018, 275: 2880-2891.
2021, Vol. 34 
