Semi-Supervised Node Classification Algorithm Based on Hierarchical Contrastive Learning
LI Yaqi1, WANG Jie2, WANG Feng1, LIANG Jiye1
1. School of Computer and Information Technology, Shanxi University, Taiyuan 030006; 2. College of Computer Science and Technology, Taiyuan University of Science and Technology, Taiyuan 030024
Abstract Most graph contrastive learning methods for semi-supervised node classification obtain two views by cumbersome data augmentation. Moreover, the above data augmentation inevitably changes the graph semantic information, limiting the efficiency and applicability of the existing graph contrastive learning methods. Therefore, a semi-supervised node classification algorithm based on hierarchical contrastive learning is proposed in this paper. In the proposed algorithm, graph data augmentation is unnecessary and the representations of different hierarchies of the graph neural network are learned as contrasted views to alleviate the tedious search and the semantic destruction. In addition, a semi-supervised contrastive loss is designed, and a small amount of labeled information and a large amount of unlabeled information are effectively utilized to provide rich supervised signals and improve the node representations. Finally, node classification experiments on four benchmark datasets validate the effectiveness of the proposed algorithm.
Fund:National Natural Science Foundation of China(No.62276158,U21A20473)
Corresponding Authors:
LIANG Jiye, Ph.D., professor. His research interests include data mining, machine learning, big data analysis and artificial intelligence.
About author:: LI Yaqi, master student. Her research interests include data mining and machine learning.WANG Jie, Ph.D., lecturer. His research interests include data mining and machine learning.WANG Feng, Ph.D., associate professor. Her research interests include feature selection, granular computing and machine lear-ning.
[1] LECUN Y, BENGIO Y, HINTON G. Deep Learning. Nature, 2015, 521(7553): 436-444. [2] 余凯,贾磊,陈雨强,等.深度学习的昨天、今天和明天.计算机研究与发展, 2013, 50(9): 1799-1804. (YU K, JIA L, CHEN Y Q, et al. Deep Learning: Yesterday, Today, and Tomorrow. Journal of Computer Research and Development, 2013, 50(9): 1799-1804.) [3] 周志华. 基于分歧的半监督学习.自动化学报, 2013, 39(11): 1871-1878. (ZHOU Z H. Disagreement-Based Semi-Supervised Learning. Acta Automatica Sinica, 2013, 39(11): 1871-1878.) [4] 刘建伟,刘媛,罗雄麟.半监督学习方法.计算机学报, 2015, 38(8): 1592-1617. (LIU J W, LIU Y, LUO X L. Semi-Supervised Learning Methods. Chinese Journal of Computers, 2015, 38(8): 1592-1617.) [5] KINGMA D P, REZENDE D J, MOHAMED S, et al. Semi-Supervised Learning with Deep Generative Models // Proc of the 27th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2014: 3581-3589. [6] BENNETT K P, DEMIRIZ A. Semi-Supervised Support Vector Machines // Proc of the 11th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 1998: 368-374. [7] BLUM A, CHAWLA S. Learning from Labeled and Unlabeled Data Using Graph Mincuts// Proc of the 18th International Conference on Machine Learning. San Francisco, USA: Morgan Kaufmann, 2001: 19-26. [8] 李明,杨艳屏,占惠融.基于局部聚类与图方法的半监督学习算法.自动化学报, 2010, 36(12): 1655-1660. (LI M, YANG Y P, ZHAN H R. Semi-Supervised Learning Based on Graph and Local Quick Shift. Acta Automatica Sinica, 2010, 36(12): 1655-1660.) [9] BLUM A, MITCHELL T. Combining Labeled and Unlabeled Data with Co-training// Proc of the 11th Annual Conference on Computational Learning Theory. New York, USA: ACM, 1998: 92-100. [10] SOHN K, BERTHELOT D, LI C L, et al. FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence // Proc of the 33rd International Conference on Neural Information Proce-ssing Systems. Cambridge, USA: MIT Press, 2020: 596-608. [11] BERTHELOT D, CARLINI N, GOODFELLOW I, et al. MixMa-tch: A Holistic Approach to Semi-Supervised Learning // Proc of the 32nd International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2019: 5049-5059. [12] PEROZZI B, AL-RFOU R, SKIENA S. DeepWalk: Online Lear-ning of Social Representations// Proc of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2014: 701-710. [13] DEFFERRARD M, BRESSON X, VANDERGHEYNST P. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering // Proc of the 29th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2016: 3844-3852. [14] VELIČKOVIĆ P, CUCURULL G, CASANOVA A, et al. Graph Attention Networks[C/OL].[2023-05-28]. https://arxiv.org/pdf/1710.10903.pdf. [15] ZHU X J, GHAHRAMANI Z, LAFFERTY J D. Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions// Proc of the 20th International Conference on Machine Learning.New York, USA: ACM, 2003: 912-919. [16] SUN K, LIN Z C, ZHU Z X. Multi-stage Self-Supervised Learning for Graph Convolutional Networks on Graphs with Few Labeled Nodes. Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020, 34(4): 5892-5899. [17] CARON M, BOJANOWSKI P, JOULIN A, et al. Deep Clustering for Unsupervised Learning of Visual Features// Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 139-156. [18] YOU Y N, CHEN T L, WANG Z Y, et al. When Does Self-Supervision Help Graph Convolutional Networks[C/OL]. [2023-05-28]. http://proceedings.mlr.press/v119/you20a/you20a.pdf. [19] LI Q M, HAN Z C, WU X M. Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning. Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018, 32(1): 3538-3545. [20] KIPF T N, WELLING M.Semi-Supervised Classification with Graph Convolutional Networks[C/OL]. [2023-05-28].https://arxiv.org/pdf/1609.02907.pdf. [21] XU M H, WANG H, NI B B, et al. Self-Supervised Graph-Level Representation Learning with Local and Global Structure// Proc of the 38th International Conference on Machine Learning. San Diego, USA: JMLR, 2021: 11548-11558. [22] 陈庆宇,季繁繁,袁晓彤.基于伪孪生网络双层优化的对比学习.模式识别与人工智能, 2022, 35(10): 928-938. (CHEN Q Y, JI F F, YUAN X T. Contrastive Learning Based on Bilevel Optimization of Pseudo Siamese Networks. Pattern Recognition and Artificial Intelligence, 2022, 35(10): 928-938.) [23] 李超,孙国义,闫页宇,等.基于拓扑信息和属性信息协同对比的自监督异质图神经网络模型.模式识别与人工智能, 2023, 36(4): 287-299. (LI C, SUN G Y, YAN Y Y, et al. Self-Supervised Heteroge-neous Graph Neural Network Model Based on Collaborative Con-trastive Learning of Topology Information and Attribute Information. Pattern Recognition and Artificial Intelligence, 2023, 36(4): 287-299.) [24] XIE Y C, XU Z, ZHANG J T, et al. Self-Supervised Learning of Graph Neural Networks: A Unified Review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(2): 2412-2429. [25] VELIČKOVIĆ P, FEDUS W, HAMILTON W L, et al. Deep Graph Infomax[C/OL].[2023-05-28]. https://arxiv.org/pdf/1809.10341.pdf. [26] HASSANI K, KHASAHMADI A H. Contrastive Multi-view Representation Learning on Graphs// Proc of the 37th International Conference on Machine Learning. San Diego, USA: JMLR, 2020:4116-4126. [27] PENG Z, HUANG W B, LUO M N, et al. Graph Representation Learning via Graphical Mutual Information Maximization// Proc of the 29th World Wide Web Conference. New York, USA: ACM, 2020: 259-270. [28] ZHU Y Q, XU Y C, YU F, et al. Deep Graph Contrastive Representation Learning[C/OL].[2023-05-28]. https://arxiv.org/pdf/2006.04131.pdf. [29] ZHU Y Q, XU Y C, YU F, et al. Graph Contrastive Learning with Adaptive Augmentation// Proc of the 30th World Wide Web Conference. New York, USA: ACM, 2021: 2069-2080. [30] SURESH S, LI P, HAO C, et al. Adversarial Graph Augmentation to Improve Graph Contrastive Learning // Proc of the 34th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2021: 15920-15933. [31] CHU G Y, WANG X, CHUAN S, et al. CuCo: Graph Representation with Curriculum Contrastive Learning// Proc of the 30th International Joint Conference on Artificial Intelligence. San Francisco, USA: Morgan Kaufmann, 2021: 2300-2306. [32] ZHU Y, GUO J H, WU F, et al. RoSA: A Robust Self-Aligned Framework for Node-Node Graph Contrastive Learning// Proc of the 31st International Joint Conference on Artificial Intelligence. San Francisco, USA: Morgan Kaufmann, 2022: 3795-3801. [33] SEN P, NAMATA G, BILGIC M, et al. Collective Classification in Network Data. AI Magazine, 2008, 29(3): 93-106. [34] SHCHUR O, MUMME M, BOJCHEVSKI A, et al. Pitfalls of Graph Neural Network Evaluation[C/OL].[2023-05-28]. https://arxiv.org/abs/1811.05868. [35] YANG Z L, COHEN W W, SALAKHUDINOV R. Revisiting Semi-Supervised Learning with Graph Embeddings// Proc of the 33rd International Conference on Machine Learning. San Diego, USA:JMLR, 2016: 40-48. [36] VAN DER MAATEN L, HIMTON G. Visualizing Data Using t-SNE. Journal of Machine Learning Research, 2008, 9: 2579-2605.
2023, Vol. 36 
