Sequential Recommendation Model Based on Smoothing Graph Masked Encoder
LIU Yang1, XIA Hongbin1,2, LIU Yuan1,2
1. School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122; 2. Jiangsu Key University Laboratory of Software and Media Tech-nology under Human-Computer Cooperation, Jiangnan Univer-sity, Wuxi 214122
Abstract Aiming at the performance degradation problem of existing sequential recommendation models caused by label sparsity and user data noise, a sequential recommendation model based on smoothing graph masked encoder(SGMERec) is proposed. Firstly, a data smoothing encoder is designed to process the data, improve data quality and reduce the negative impact of extreme values and data noise. Secondly, a graph masked encoder is designed to adaptively extract transformation information from global items and a relational graph is constructed to help the model complete the missing label data, thereby enhancing the ability to deal with issues of label scarcity. Finally, batch normalization is employed to normalize the input distribution of each neural network layer. Thus, the stability of input distribution for each layer is guaranteed and the proportion of scarce labels in user sequences is reduced. Experimental results on three real datasets indicate the performance improvement of SGMERec.
Fund:National Natural Science Foundation of China(No.61972182)
Corresponding Authors:
XIA Hongbin, Ph.D., professor. His research interests include personalized recommendation and natural language processing.
About author:: LIU Yang, Master student. His research interests include recommendation systems and deep learning. LIU Yuan, Ph.D., professor. His research interests include network security and social network.
LIU Yang,XIA Hongbin,LIU Yuan. Sequential Recommendation Model Based on Smoothing Graph Masked Encoder[J]. Pattern Recognition and Artificial Intelligence, 2024, 37(6): 525-537.
[1] YANG Y H, HUANG C, XIA L H, et al. Multi-behavior Hyper-graph-Enhanced Transformer for Sequential Recommendation // Proc of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2022: 2263-2274. [2] YE W W, WANG S Q, CHEN X, et al. Time Matters: Sequential Recommendation with Complex Temporal Information // Proc of the 43rd International ACM SIGIR Conference on Research and Deve-lopment in Information Retrieval. New York, USA: ACM, 2020: 1459-1468. [3] WANG J L, LOUCA R, HU D N, et al. Time to Shop for Valentine's Day: Shopping Occasions and Sequential Recommendation in E-commerce // Proc of the 13th International Conference on Web Search and Data Mining. New York, USA: ACM, 2020: 645-653. [4] WEI Y W, WANG X, NIE L Q, et al. MMGCN: Multi-modal Graph Convolution Network for Personalized Recommendation of Micro-Video // Proc of the 27th ACM International Conference on Multimedia. New York, USA: ACM, 2019: 1437-1445. [5] WANG Z Y, WEI W, CONG G, et al. Global Context Enhanced Graph Neural Networks for Session-Based Recommendation // Proc of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM, 2020: 169-178. [6] ZHU Y Q, XU Y C, YU F, et al. Graph Contrastive Learning with Adaptive Augmentation // Proc of the Web Conference. New York, USA: ACM, 2021: 2069-2080. [7] JING M Y, ZHU Y M, ZANG T Z, et al. Graph Contrastive Lear-ning with Adaptive Augmentation for Recommendation // Proc of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Berlin, Germany: Springer, 2022: 590-605. [8] TIAN C X, LI Y X, LI Y L, et al. Learning to Denoise Unreliable Interactions for Graph Collaborative Filtering // Proc of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM, 2022: 122-132. [9] XIA L H, HUANG C, ZHANG C X. Self-Supervised Hypergraph Transformer for Recommender Systems // Proc of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2022: 2100-2109. [10] CHEN J W, DONG H D, WANG X, et al. Bias and Debias in Recommender System: A Survey and Future Directions. ACM Transactions on Information Systems, 2023, 41(3). DOI: 10.1145/3564284. [11] QIU R H, HUANG Z, YIN H Z, et al. Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation // Proc of the 15th ACM International Conference on Web Search and Data Mining. New York, USA: ACM, 2022: 813-823. [12] LIU Z W, CHEN Y J, LI J, et al. Contrastive Self-Supervised Sequential Recommendation with Robust Augmentation[C/OL].[2024-05-20]. https://arxiv.org/pdf/2108.06479. [13] XIE X, SUN F, LIU Z Y, et al. Contrastive Learning for Sequential Recommendation // Proc of the IEEE 38th International Conference on Data Engineering. Washington, USA: IEEE, 2022: 1259-1273. [14] YE Y W, XIA L H, HUANG C. Graph Masked Autoencoder for Sequential Recommendation[C/OL]. [2024-05-20].https://arxiv.org/pdf/2305.04619. [15] WANG S J, HU L, WANG Y, et al. Sequential Recommender Systems: Challenges, Progress and Prospects[C/OL].[2024-05-20]. https://arxiv.org/pdf/2001.04830. [16] KIM D, PARK C, OH J, et al. Convolutional Matrix Factorization for Document Context-Aware Recommendation // Proc of the 10th ACM Conference on Recommender Systems. New York, USA: ACM, 2016: 233-240. [17] WU S, TANG Y Y, ZHU Y Q, et al. Session-Based Recommendation with Graph Neural Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1): 346-353. [18] CHANG J X, GAO C, ZHENG Y, et al. Sequential Recommendation with Graph Neural Networks // Proc of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM, 2021: 378-387. [19] ZHAO X Y, XIA L, ZHANG L, et al. Deep Reinforcement Lear-ning for Page-Wise Recommendations // Proc of the 12th ACM Conference on Recommender Systems. New York, USA: ACM, 2018: 95-103. [20] YANG Y H, HUANG C, XIA L H, et al. Debiased Contrastive Learning for Sequential Recommendation // Proc of the ACM Web Conference. New York, USA: ACM, 2023: 1063-1073. [21] ZHOU K, WANG H, WEN J K, et al. Enhancing Multi-view Smoothness for Sequential Recommendation Models. ACM Transactions on Information Systems, 2023, 41(4). DOI: 10.1145/35824. [22] HIDASI B, KARATZOGLOU A, BALTRUNAS L, et al. Session-Based Recommendations with Recurrent Neural Networks[C/OL].[2024-05-20]. https://arxiv.org/pdf/1511.06939. [23] ZHOU X, SUN Z, GUO G B, et al. Modelling Temporal Dynamics and Repeated Behaviors for Recommendation // Proc of the 24th Pacific-Asia Conference on Knowledge Discovery and Data Mining. Berlin, Germany: Springer, 2020: 181-193. [24] COOIJMANS T, BALLAS N, LAURENT C, et al. Recurrent Batch Normalization[C/OL].[2024-05-20]. https://arxiv.org/pdf/1603.09025. [25] SUN P J, WU L, WANG M. Attentive Recurrent Social Reco-mmendation // Proc of the 41st International ACM SIGIR Confe-rence on Research and Development in Information Retrieval. New York, USA: ACM, 2018: 185-194. [26] WU L W, LI S Q, HSIEH C J, et al. SSE-PT: Sequential Reco-mmendation via Personalized Transformer // Proc of the 14th ACM Conference on Recommender Systems. New York, USA: ACM, 2020: 328-337. [27] LI J, REN P J, CHEN Z M, et al. Neural Attentive Session-Based Recommendation // Proc of the ACM Conference on Information and Knowledge Management. New York, USA: ACM, 2017: 1419-1428. [28] KANG W C, MCAULEY J. Self-Attentive Sequential Recommendation // Proc of the IEEE International Conference on Data Mining. Washington, USA: IEEE, 2018: 197-206. [29] SUN F, LIU J, WU J, et al. BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer // Proc of the 28th ACM International Conference on Information and Knowledge Management. New York, USA: ACM, 2019: 1441-1450. [30] WANG J L, DING K Z, HONG L J, et al. Next-Item Recommendation with Sequential Hypergraphs // Proc of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM, 2020: 1101-1110. [31] WANG C Y, MA W Z, CHEN C, et al. Sequential Recommendation with Multiple Contrast Signals. ACM Transactions on Information Systems, 2023, 41(1). DOI: 10.1145/35226. [32] CHEN Y J, LIU Z W, LI J, et al. Intent Contrastive Learning for Sequential Recommendation // Proc of the ACM Web Conference. New York, USA: ACM, 2022: 2172-2182.
2024, Vol. 37 
