Sequential Recommendation Model Based on Temporal Convolution Attention Neural Network
DU Yongping1, NIU Jinyu1, WANG Lulin2, YAN Rui1
1. Faculty of Information Technology, Beijing University of Technology, Beijing 100124; 2. State Key Laboratory of Computer Architecture, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100086
Abstract Sequential recommendation task aims to dynamically model user interests based on user-item interaction records for item recommendation. In sequential recommendation models, user behaviors are usually modeled as interests. The models only consider the order of user behaviors while ignoring the time interval information between users. In this paper, the time interval information of behavior sequences is taken as an important factor for prediction. A temporal convolution attention neural network model(TCAN) is proposed. In the word embedding layer, the sequential position information and time interval information are introduced, and a temporal convolutional network is designed to model the position information to obtain user's long-term preference features. In addition, the two-layer self-attention mechanism is adopted to model the association between items in the user's short-term behavior sequence, and the time interval information is fused to obtain the user's short-term interest. Finally, the global information of the training data is introduced through pre-training to improve the model recommendation performance. Experiments on three datasets show that the proposed model effectively improves recommendation performance.
Fund:National Key R&D Program of China(No.2019YFC1906002), Beijing Natural Science Foundation(No.4212013)
Corresponding Authors:DU Yongping, Ph.D., professor. Her research interests include information retrieval, information extraction and natural language processing.
About author:: NIU Jinyu, master student. His research interests include recommendation system and knowledge distillation. WANG Lulin, master, engineer. His research interests include intelligent information processing and recommendation system. YAN Rui, master student. His research interests include natural language processing.
DU Yongping,NIU Jinyu,WANG Lulin等. Sequential Recommendation Model Based on Temporal Convolution Attention Neural Network[J]. Pattern Recognition and Artificial Intelligence, 2022, 35(5): 472-480.
[1] RENDLE S, FREUDENTHALER C, GANTNER Z, et al. BPR: Bayesian Personalized Ranking from Implicit Feedback // Proc of the 25th Conference on Uncertainty in Artificial Intelligence. Arlington, USA: AUAI Press, 2009: 452-461. [2] SHANI G, HECKERMAN D, BRAFMAN R I, et al. An MDP-Based Recommender System. Journal of Machine Learning Research, 2005, 6: 1265-1295. [3] HIDASI B, KARATZOGLOU A. Recurrent Neural Networks with Top-k Gains for Session-Based Recommendations // Proc of the 27th ACM International Conference on Information and Knowledge Ma-nagement. New York, USA: ACM, 2018: 843-852. [4] TANG J X, WANG K. Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding // Proc of the 11th ACM International Conference on Web Search and Data Mining. New York, USA: ACM, 2018: 565-573. [5] YOU J X, WANG Y C, PAL A, et al. Hierarchical Temporal Con-volutional Networks for Dynamic Recommender Systems // Proc of the World Wide Web Conference. New York, USA: ACM, 2019: 2236-2246. [6] KANG W C, MCAULEY J. Self-Attentive Sequential Recommendation // Proc of the IEEE International Conference on Data Mining. Washington, USA: IEEE, 2018: 197-206. [7] 王鸿伟,过敏意.刻画长短期用户兴趣的基于会话的推荐系统.中国科学(信息科学), 2020, 50(12): 1867-1881. (WANG H W, GUO M Y. Recurrent Memory Networks: Modeling Long Short-Term User Preferences for Session-Based Recommendation. Scientia Sinica(Informationis), 2020, 50(12): 1867-1881.) [8] 冯永,张备,强保华,等.MN-HDRM:长短兴趣多神经网络混合动态推荐模型.计算机学报, 2019, 42(1): 16-28. (FENG Y, ZHANG B, QIANG B H, et al. MN-HDRM: A Novel Hybrid Dynamic Recommendation Model Based on Long-Short-Term Interests Multiple Neural Networks. Chinese Journal of Computers, 2019, 42(1): 16-28.) [9] 阎世宏,马为之,张敏,等.结合用户长短期兴趣的深度强化学习推荐方法.中文信息学报, 2021, 35(8): 107-116. (YAN S H, MA W Z, ZHANG M, et al. Reinforcement Learning with User Long-Term and Short-Term Preference for Personalized Recommendation. Journal of Chinese Information Processing, 2021, 35(8): 107-116.) [10] WU C Y, AHMED A, BEUTEL A, et al. Recurrent Recommender Networks // Proc of the 10th ACM International Conference on Web Search and Data Mining. New York, USA: ACM, 2017: 495-503. [11] LIU Q, ZENG Y F, MOKHOSI R, et al. STAMP: Short-Term Attention/Memory Priority Model for Session-Based Recommendation // Proc of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2018: 1831-1839. [12] 袁涛,牛树梓,李会元.一种基于CW-RNN的多时间尺度序列建模推荐算法.中文信息学报, 2020, 34(6): 97-105. (YUAN T, NIU S Z, LI H Y. A Multi-scale Temporal Dynamic Model for Sequential Recommendation with Clockwork RNN. Journal of Chinese Information Processing, 2020, 34(6): 97-105.) [13] LI J C, WANG Y J, MCAULEY J. Time Interval Aware Self-Attention for Sequential Recommendation // Proc of the 13th International Conference on Web Search and Data Mining. New York, USA: ACM, 2020: 322-330. [14] JI W D, WANG K Q, WANG X L, et al. Sequential Recommender via Time-Aware Attentive Memory Network // Proc of the 29th ACM International Conference on Information and Knowledge Management. New York, USA: ACM, 2020: 565-574. [15] ZHOU H C, TAN Q Y, HUANG X, et al. Temporal Augmented Graph Neural Networks for Session-Based Recommendations // Proc of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM, 2021: 1798-1802. [16] HSU C, LI C T. RetaGNN: Relational Temporal Attentive Graph Neural Networks for Holistic Sequential Recommendation // Proc of the Web Conference. New York, USA: ACM, 2021: 2968-2979. [17] VASWANI A, SHAZEER N, PARMAR N, et al. Attention Is All You Need // Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2017: 6000-6010. [18] SHAW P, USZKOREIT J, VASWANI A. Self-Attention with Relative Position Representations // Proc of the Conference of the North American Chapter of the Association for Computational Linguistics(Human Language Technologies). Stroudsburg, USA: ACL, 2018: 464-468. [19] HE R N, KANG W C, MCAULEY J. Translation-Based Reco-mmendation // Proc of the 11th ACM Conference on Recommender Systems. New York, USA: ACM, 2017: 161-169.
2022, Vol. 35 
