Self-Supervised Heterogeneous Graph Neural Network Model Based on Collaborative Contrastive Learning of Topology Information and Attribute Information
LI Chao1, SUN Guoyi1, YAN Yeyu1, DUAN Hua3, ZENG Qingtian2
1. College of Electronic and Information Engineering, Shandong University of Science and Technology, Qingdao 266590; 2. College of Mathematics and Systems Science, Shandong University of Science and Technology, Qingdao 266590; 3. College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao 266590
Abstract：The complex structure and rich semantics of heterogeneous graphs can be fully explored by heterogeneous graph neural network models. However, there is mutual interference between attribute information and topology information in the model construction process, resulting in weakened expression capability. Therefore, a self-supervised heterogeneous graph neural network model based on collaborative contrastive learning of topology information and attribute information is proposed. Firstly, the representation of the target nodes is learned from both topological and attribute perspectives. Then, the collaborative contrastive algorithm is employed to optimize the node representation from both perspectives, reducing the interference between topology information and attribute information. Additionally, a positive sample generation method combining the number of meta-paths and node topology similarity is proposed in the self-supervised training process of the model. The experiments on real datasets demonstrate the superior performance of the proposed model. The model code can be found at https://github.com/sun281210/HGTA.
李超, 孙国义, 闫页宇, 段华, 曾庆田. 基于拓扑信息和属性信息协同对比的自监督异质图神经网络模型[J]. 模式识别与人工智能, 2023, 36(4): 287-299.
LI Chao, SUN Guoyi, YAN Yeyu, DUAN Hua, ZENG Qingtian. Self-Supervised Heterogeneous Graph Neural Network Model Based on Collaborative Contrastive Learning of Topology Information and Attribute Information. Pattern Recognition and Artificial Intelligence, 2023, 36(4): 287-299.
[1] HAMILTON W L, YING R, LESKOVEC J. Inductive Representation Learning on Large Graphs // Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2017: 1025-1035. [2] GAO X Y, FENG F L, HUANG H Y, et al. Food Recommendation with Graph Convolutional Network. Information Sciences, 2022, 584: 170-183. [3] DAVIS A P, GRONDIN C J, JOHNSON R J, et al. The Comparative Toxicogenomics Database: Update 2019. Nucleic Acids Research, 2019, 47(D1): D948-D954. [4] WU Z H, PAN S R, LONG G D, et al. Graph Wavenet for Deep Spatial-Temporal Graph Modeling // Proc of the 28th International Joint Conference on Artificial Intelligence. San Francisco, USA: IJCAI, 2019: 1907-1913. [5] SUN Y Z, HAN J W.Mining Heterogeneous Information Networks: A Structural Analysis Approach. ACM SIGKDD Explorations Newsletter, 2012, 14(2): 20-28. [6] ZHAO J N, WANG X, SHI C, et al. Network Schema Preserving Heterogeneous Information Network Embedding // Proc of the 29th International Joint Conference on Artificial Intelligence. San Francisco, USA: IJCAI, 2021: 1366-1372. [7] WANG X, JI H Y, SHI C, et al. Heterogeneous Graph Attention Network // Proc of the World Wide Web Conference. New York, USA: ACM, 2019: 2022-2032. [8] FU X Y, ZHANG J N, MENG Z Q, et al. MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding // Proc of the Web Conference. New York, USA: ACM, 2020: 2331-2341. [9] YUN S, JEONG M, KIM R, et al. Graph Transformer Networks // Proc of the 33rd International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2019: 960-970. [10] LI C, YAN Y Y, FU J H, et al. HetReGAT-FC: Heterogeneous Residual Graph Attention Network via Feature Completion. Information Sciences, 2023, 632: 424-438. [11] HU W H, LIU B W, GOMES J, et al. Strategies for Pre-training Graph Neural Networks[C/OL].[2022-12-10]. https://arxiv.org/pdf/1905.12265v3.pdf. [12] 陈庆宇,季繁繁,袁晓彤.基于伪孪生网络双层优化的对比学习.模式识别与人工智能, 2022, 35(10): 928-938. (CHEN Q Y, JI F F, YUAN X T.Contrastive Learning Based on Bilevel Optimization of Pseudo Siamese Networks. Pattern Recognition and Artificial Intelligence, 2022, 35(10): 928-938.) [13] ZHANG C X, SONG D J, HUANG C, et al. Heterogeneous Graph Neural Network // Proc of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2019: 793-803. [14] VELIČKOVIĆ P, FEDUS W, HAMILTON W L, et al. Deep Graph Infomax[C/OL].[2022-12-10]. https://arxiv.org/pdf/1209.10341.pdf. [15] REN Y X, LIU B, HUANG C, et al. Heterogeneous Deep Graph Infomax[C/OL].[2022-12-10]. https://arxiv.org/pdf/1911.08538.pdf. [16] JING B Y, PARK C, TONG H H.HDMI: High-Order Deep Multi-plex Infomax // Proc of the World Wide Web Conference. New York, USA: ACM, 2021: 2414-2424. [17] PARK C, HAN J W, YU H.Deep Multiplex Graph Infomax: Atten-tive Multiplex Network Embedding Using Global Information. Knowledge-Based Systems, 2020, 197. DOI: 10.1016/j.knosys.2020.105861. [18] ZHU Y Q, XU Y C, YU F, et al. Deep Graph Contrastive Representation Learning[C/OL].[2022-12-10]. https://arxiv.org/pdf/2006.04131.pdf. [19] YOU Y N, CHEN T L, SUI Y D, et al. Graph Contrastive Lear-ning with Augmentations // Proc of the 34th International Confe-rence on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2020: 5812-5823. [20] QIU J Z, CHEN Q B, DONG Y X, et al. GCC: Graph Contrastive Coding for Graph Neural Network Pre-training // Proc of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2020: 1150-1160. [21] HE K M, FAN H Q, WU Y X, et al. Momentum Contrast for Unsupervised Visual Representation Learning // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 9726-9735. [22] ZHU Y Q, XU Y C, YU F, et al. Graph Contrastive Learning with Adaptive Augmentation // Proc of the Web Conference. New York, USA: ACM, 2021: 2069-2080. [23] WANG X, LIU N, HAN H, et al. Self-Supervised Heterogeneous Graph Neural Network with Co-contrastive Learning // Proc of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2021: 1726-1736. [24] WANG Z H, LI Q, YU D H, et al. Heterogeneous Graph Contrastive Multi-view Learning[C/OL].[2022-12-10]. https://arxiv.org/pdf/2210.00248.pdf. [25] YU J X, LI X.Heterogeneous Graph Contrastive Learning with Meta-Path Contexts and Weighted Negative Samples[C/OL]. [2022-12-10].https://arxiv.org/pdf/2212.13847.pdf. [26] YANG L, CHEN Z Y, GU J H, et al. Dual Self-Paced Graph Convolutional Network: Towards Reducing Attribute Distortions Induced by Topology // Proc of the 28th International Joint Confe-rence on Artificial Intelligence. San Francisco, USA: IJCAI, 2019: 4062-4069. [27] YANG L, ZHOU W M, PENG W H, et al. Graph Neural Networks Beyond Compromise Between Attribute and Topology // Proc of the ACM Web Conference. New York, USA: ACM, 2022: 1127-1135. [28] JIN W, DERR T, WANG Y Q, et al. Node Similarity Preserving Graph Convolutional Networks // Proc of the 14th ACM International Conference on Web Search and Data Mining. New York, USA: ACM, 2021: 148-156. [29] SHI C, YU P S.Heterogeneous Information Network Analysis and Applications. Berlin, Germany: Springer, 2017. [30] DONG Y X, CHAWLA N V, SWAMI A.Metapath2vec: Scalable Representation Learning for Heterogeneous Networks // Proc of the 23rd ACM SIGKDD International Conference on Knowledge Disco-very and Data Mining. New York, USA: ACM, 2017: 135-144. [31] LI X, DING D H, KAO B, et al. Leveraging Meta-Path Contexts for Classification in Heterogeneous Information Networks // Proc of the IEEE 37th International Conference on Data Engineering. Washington, USA: IEEE, 2021: 912-923. [32] SHI C, HU B B, ZHAO W N, et al. Heterogeneous Information Network Embedding for Recommendation. IEEE Transactions on Knowledge and Data Engineering, 2019, 31(2): 357-370.
2023, Vol. 36 
