Multi-label Learning for Non-equilibrium Labels Completion in Neighborhood Labels Space
CHENG Yusheng1,2, ZHAO Dawei1, QIAN Kun1
1.School of Computer and Information, Anqing Normal University, Anqing 246133
2.University Key Laboratory of Intelligent Perception and Computing of Anhui Province, Anqing Normal University, Anqing 246133
The correlation between labels are studied through the related information about the marked labels. However, the influence of the relationship between unmarked and marked labels on the quality of the multi-label set is not taken into account. Inspired by k-nearest neighbors(KNN), a non-equilibrium labels completion of neighboring labels space(NeLC-NLS) is proposed to improve the quality of the neighboring label space and the performance of the multi-label classification. Firstly, the information entropy between labels is utilized to measure the strength of the relationship between labels, and the confidence matrix of the basic label is obtained. Then, the confidence matrix of non-equilibrium labels containing more information is obtained via the proposed non-equilibrium label confidence matrix. Secondly, the similarity of samples is measured in the feature space and the k-nearest neighbors are obtained. Then, the non-equilibrium labels completion matrix is employed to calculate the label completion matrix of the neighboring labels space. Finally, the extreme learning machine is adopted as a linear classifier. The experimental results of the proposed algorithm on 8 public multi-label datasets show that NeLC-NLS is superior to other multi-label learning algorithms. The effectiveness of NeLC-NLS is further illustrated by using hypothesis testing and stability analysis.
[1] ZHANG M L, ZHOU Z H. Multi-label Learning // SAMMUT C, WEBB G I, eds. Encyclopedia of Machine Learning and Data Mi-ning. Berlin, Germany: Springer, 2017: 875-881.
[2] BOUTELL M R, LUO J B, SHEN X P, et al. Learning Multi-label Scene Classification. Pattern Recognition, 2004, 37(9): 1757-1771.
[3] ZHANG M L, ZHOU Z H. Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization. IEEE Transactions on Knowledge and Data Engineering, 2006, 18(10): 1338-1351.
[4] ZHANG M L, ZHOU Z H. ML-KNN: A Lazy Learning Approach to Multi-label Learning. Pattern Recognition, 2007, 40(7): 2038-2048.
[5] READ J, PFAHRINGER B, HOLMES G, et al. Classifier Chains for Multi-label Classification. Machine Learning, 2011, 85(3). DOI: https://doi.org/10.1007/s10994-011-5256-5.
[6] FÜRNKRANZ J, HÜLLERMEIER E, MENCIA E L, et al. Multilabel Classification via Calibrated Label Ran-king. Machine Learning, 2008, 73(2): 133-153.
[7] TSOUMAKAS G, KATAKIS I, VLAHAVAS I. Random k-Labelsets for Multilabel Classification. IEEE Transactions on Knowledge and Data Engineering, 2011, 23(7): 1079-1089.
[8] ELISSEEFF A, WESTON J. A Kernel Method for Multi-labelled Classification // DIETTERICH T G, BECHER S, GHAHRAMANI Z, eds. Advances in Neural Information Processing Systems 14. Cambridge, USA: The MIT Press, 2002: 681-687.
[9] ZHANG Y, YEUNG D Y. Multilabel Relationship Learning. ACM Transactions on Knowledge Discovery from Data, 2013, 7(2). DOI: 10.1145/2499907.2499910.
[10] SHANNON C E. A Mathematical Theory of Communication. The Bell System Technical Journal, 1948, 27(3): 379-423.
[11] 张振海,李士宁,李志刚.一种基于相关信息熵的多标签分类算法.西北工业大学学报, 2012, 30(6): 968-973.
(ZHANG Z H, LI S N, LI Z G. A Multi-label Classification Algorithm Using Correlation Information Entropy. Journal of Northwes-tern Polytechnical University, 2012, 30(6): 968-973.)
[12] LEE J, KIM H, KIM N R, et al. An Approach for Multi-label Classification by Directed Acyclic Graph with Label Correlation Maximization. Information Sciences, 2016, 351: 101-114.
[13] HUANG S J, YU Y, ZHOU Z H. Multi-label Hypothesis Reuse // Proc of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2012: 525-533.
[14] 张敏灵.一种新型多标记懒惰学习算法.计算机研究与发展, 2012, 49(11): 2271-2282.
(ZHANG M L. An Improved Multi-label Lazy Learning Approach. Journal of Computer Research and Development, 2012, 49(11): 2271-2282.)
[15] YOUNES Z, ABDALLAH F, DENOEUX T. Multi-label Classification Algorithm Derived from K-Nearest Neighbor Rule with Label Dependencies[C/OL].[2018-03-15]. https:// ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7080359.
[16] GWEON H, SCHONLAU M, STEINER S. Nearest Labelset Using Double Distances for Multi-label Classification[C/OL]. [2018-03-15]. https://arxiv.org/pdf/1702.04684.pdf.
[17] HUANG G, HUANG G B, SONG S J, et al. Trends in Extreme Learning Machines: A Review. Neural Networks, 2015, 61: 32-48.
[18] HUANG G B. An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels. Cognitive Computation, 2014, 6(3): 376-390.
[19] DING S F, XU X Z, NIE R. Extreme Learning Machine and Its Applications. Neural Computing and Applications, 2014, 25(3/4): 549-556.
[20] LIU Y, WEN K W, GAO Q X, et al. SVM Based Multi-label Learning with Missing Labels for Image Annotation. Pattern Recognition, 2018, 78: 307-317.
[21] PIZZUTI C. A Multi-objective Genetic Algorithm for Community Detection in Networks // Proc of the 21st IEEE International Conference on TOOLS with Artificial Intelligence. Washington, USA: IEEE, 2009: 379-386.
[22] ZHANG M L, ZHOU Z H. A Review on Multi-label Learning Algorithms. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(8): 1819-1837.
[23] 邓万宇,郑庆华,陈 琳,等.神经网络极速学习方法研究.计算机学报, 2010, 33(2): 279-287.
(DENG W Y, ZHENG Q H, CHEN L, et al. Research on Extreme Learning of Neural Networks. Chinese Journal of Computers, 2010, 33(2): 279-287.)
[24] LUO F F, GUO W Z, YU Y L, et al. A Multi-label Classification Algorithm Based on Kernel Extreme Learning Machine. Neurocomputing, 2017, 260: 313-320.
[25] ZHANG M L, WU L. Lift: Multi-label Learning with Label-Speci-fic Features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(1): 107-120.
[26] DEMSˇAR J. Statistical Comparisons of Classifiers over Multiple Data Sets. Journal of Machine Learning Research, 2006, 7: 1-30.
[27] LIN Y J, LI Y W, WANG C X, et al. Attribute Reduction for Multi-label Learning with Fuzzy Rough Set. Knowledge-Based Systems, 2018, 152: 51-61.
2018, Vol. 31 
