News Recommendation Model Based on Transformer and Heterogenous Graph Neural Network
ZHANG Yupeng1, LI Xiangju1, LI Chao2, ZHAO Zhongying1
1. College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao 266590; 2. College of Electronic and Information Engineering, Shandong University of Science and Technology, Qingdao 266590
Abstract In most of the existing news recommendation models, it is assumed that there is strong temporal dependence among the news items browsed by users. However, noise may be introduced into temporal modeling due to the rapidity of news updates and the freedom for users to read. To solve the problem, a news recommendation model based on Transformer and heterogenous graph neural network is proposed. Different from the neural network model based on time series, Transformer is employed to model the users’ short-term interests from the recent reading history. Using heterogenous graph neural networks, users’ long-term interests and candidate news representations are modeled by capturing the high-order relationship information between users and news. Meanwhile, a long and short-term interests aware mechanism is designed to adaptively adjust the importance of users’ long-term and short-term interests in news recommendation. Experiments on a real-world dataset demonstrate the effectiveness of the proposed model.
Fund:Supported by National Natural Science Foundation of China(No.62072288,61702306)
Corresponding Authors:
LI Xiangju, Ph.D., lecturer. Her research interests include natural language processing, sentiment analysis and recommendation algorithms.
About author:: ZHANG Yupeng, master student. His research interests include machine learning and recommendation system. LI Chao, Ph.D., associate professor. His research interests include network representation learning, social network analysis and re-commendation algorithm.ZHAO Zhongying, Ph.D., associate professor. Her research interests include social network analysis, data mining and recommendation system.
ZHANG Yupeng,LI Xiangju,LI Chao等. News Recommendation Model Based on Transformer and Heterogenous Graph Neural Network[J]. Pattern Recognition and Artificial Intelligence, 2022, 35(9): 839-848.
[1] DAS A S, DATAR M, GARG A, et al. Google News Personalization: Scalable Online Collaborative Filtering // Proc of the 16th International Conference on World Wide Web. New York, USA: ACM, 2007: 271-280. [2] WANG H W, ZHANG F Z, XIE X, et al. DKN: Deep Knowledge-Aware Network for News Recommendation // Proc of the World Wide Web Conference. New York, USA: ACM, 2018: 1835-1844. [3] LI M M, WANG L C.A Survey on Personalized News Recommendation Technology. IEEE Access, 2019, 7: 145861-145879. [4] XUE H J, DAI X Y, ZHANG J B, et al. Deep Matrix Factorization Models for Recommender Systems // Proc of the 26th International Joint Conference on Artificial Intelligence. San Francisco, USA: Morgan Kaufmann, 2017: 3203-3209. [5] WANG X, HE X N, WANG M, et al. Neural Graph Collaborative Filtering // Proc of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM, 2019: 165-174. [6] BANSAL T, DAS M, BHATTACHARYYA C.Content Driven User Profiling for Comment-Worthy Recommendations of News and Blog Articles // Proc of the 9th ACM Conference on Recommender Systems. New York, USA: ACM, 2015: 195-202. [7] CHENG H T, KOC L, HARMSEN J, et al. Wide & Deep Learning for Recommender Systems // Proc of the 1st Workshop on Deep Learning for Recommender Systems. New York, USA: ACM, 2016: 7-10. [8] WU C H, WU F Z, GE S Y, et al. Neural News Recommendation with Multi-head Self-Attention // Proc of the Conference on Empirical Methods in Natural Language Processing and the 9th Internatio-nal Joint Conference on Natural Language Processing. Stroudsburg, USA: ACL, 2019: 6389-6394. [9] FAN W Q, MA Y, LI Q, et al. Graph Neural Networks for Social Recommendation // Proc of the World Wide Web Conference. New York, USA: ACM, 2019: 417-426. [10] QIU Z P, HU Y F, WU X.Graph Neural News Recommendation with User Existing and Potential Interest Modeling. ACM Transactions on Knowledge Discovery from Data, 2022, 16(5). DOI: 10.1145/3511708. [11] WU C H, WU F Z, AN M X, et al. Neural News Recommendation with Attentive Multi-view Learning // Proc of the 28th International Joint Conference on Artificial Intelligence. San Francisco, USA: Morgan Kaufmann, 2019: 3863-3869. [12] 田萱,丁琪,廖子慧,等.基于深度学习的新闻推荐算法研究综述.计算机科学与探索, 2021, 15(6): 971-998. (TIAN X, DING Q, LIAO Z H, et al. Survey on Deep Learning Based News Recommendation Algorithm. Journal of Frontiers of Computer Science and Technology, 2021, 15(6): 971-998.) [13] WU C H, WU F Z, HUANG Y F, et al. User-as-Graph: User Modeling with Heterogeneous Graph Pooling for News Recommendation // Proc of the 30th International Joint Conference on Artificial Intelligence. San Francisco, USA: Morgan Kaufmann, 2021: 1624-1630. [14] LI J, ZHU J M, BI Q W, et al. MINER: Multi-interest Matching Network for News Recommendation // Proc of the Findings of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2022: 343-352. [15] WU C H, WU F Z, QI T, et al. User Modeling with Click Prefe-rence and Reading Satisfaction for News Recommendation // Proc of the 29th International Joint Conference on Artificial Intelligence. San Francisco, USA: Morgan Kaufmann, 2020: 3023-3029. [16] OKURA S, TAGAMI Y, ONO S, et al. Embedding-Based News Recommendation for Millions of Users // Proc of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2017: 1933-1942. [17] KHATTAR D, KUMAR V, VARMA V, et al. HRAM: A Hybrid Recurrent Attention Machine for News Recommendation // Proc of the 27th ACM International Conference on Information and Knowledge Management. New York, USA: ACM, 2018: 1619-1622. [18] ZHU Q N, ZHOU X F, SONG Z L, et al. DAN: Deep Attention Neural Network for News Recommendation. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1): 5973-5980. [19] HU L M, LI C, SHI C, et al. Graph Neural News Recommendation with Long-Term and Short-Term Interest Modeling. Information Processing & Management, 2020, 57(2). DOI: 10.1016/j.ipm.2019.102142. [20] AN M X, WU F Z, WU C H, et al. Neural News Recommendation with Long-and Short-Term User Representations // Proc of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2019: 336-345. [21] WU C H, WU F Z, QI T, et al. Is News Recommendation a Sequential Recommendation Task? // Proc of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM, 2022: 2382-2386. [22] LIU S W, OUNIS I, MACDONALD C, et al. A Heterogeneous Graph Neural Model for Cold-Start Recommendation // Proc of the 43rd International ACM SIGIR Conference on Research and Deve-lopment in Information Retrieval. New York, USA: ACM, 2020: 2029-2032. [23] BLEI D M, NG A Y, JORDAN M I. Latent Dirichlet Allocation. Journal of Machine Learning Research, 2003, 3: 993-1022. [24] VASWANI A, SHAZEER N, PARMAR N, et al.Attention Is All You Need // Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2017: 6000-6010. [25] GULLA J A, ZHANG L M, LIU P, et al. The Adressa Dataset for News Recommendation // Proc of the International Conference on Web Intelligence. New York, USA: ACM, 2017: 1042-1048. [26] KINGMA D P, BA J L. Adam: A Method for Stochastic Optimization[C/OL]. [2022-04-15]. https://arxiv.org/pdf/1412.6980.pdf. [27] GUO H F, TANG R M, YE Y M, et al. DeepFM: A Factorization-Machine Based Neural Network for CTR Prediction // Proc of the 26th International Joint Conference on Artificial Intelligence. San Francisco, USA: Morgan Kaufmann, 2017: 1725-1731. [28] JI Z Y, WU M D, LIU J R, et al. Attention-Based Graph Neural Network for News Recommendation // Proc of the International Joint Conference on Neural Networks. Washington, USA: IEEE, 2021. DOI: 10.1109/IJCNN52387.2021.9534339.
2022, Vol. 35 
