Long Query User Satisfaction Analysis Based on User Behaviors
ZHU Tong, LIU Yi-Qun, RU Li-Yun, MA Shao-Ping
State Key Laboratory of Intelligent Technology and Systems,Beijing 100084 Tsinghua National Laboratory for Information Science and Technology,Beijing 100084 Department of Computer Science and Technology,Tsinghua University,Beijing 100084
Abstract Performance evaluation is one of the most important issues in web search. Long queries contain much information which describes users information demand correctly. Thus, a long query search user satisfaction detection framework is proposed. The concept of user satisfaction is defined. The relevant user behavior features in user logs are extracted which are combined with Decision Tree and SVM to identify satisfactory or unsatisfactory queries. The experimental results on large scale practical search engine data show the effectiveness of the proposed framework. Furthermore, the classification accuracies of satisfactory and unsatisfactory queries reach 86% and 70%, respectively.
[1] China Internet Network Information Center.The 25th Report in Development of Internet in China [EB/OL].[2010-01-01]. http://www.cnnic.net.cn/uploadfiles/pdf/2010/1/15/101600.pdf (in Chinese) (中国互联网络信息中心.中国互联网络发展状况统计报告[EB/OL].[2010-01-01]. http://www.cnnic.net.cn/uploadfiles/pdf/2010/1/15/101600.pdf) [2] Cockburn A,Jones S.Which Way Now? Analysing and Easing Inadequacies in WWW Navigation.International Journal of Human: Computer Studies,1996,45(1): 105-129 [3] Tauscher L,Greenberg S.How People Revisit Web Pages: Empirical Findings and Implications for the Design of History Systems.International Journal of Human: Computer Studies,1997,47(1): 97-137 [4] Yu Huijia,Liu Yiqun,Zhang Min,et al.Web Spam Identification with User Browsing Graph // Proc of the 5th Asia Information Retrieval Symposium on Information Retrieval Technology.Sapporo,Japan,2009: 38-49 [5] Agichtein E,Brill E,Dumais S.Improving Web Search Ranking by Incorporating User Behavior Information // Proc of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.Seattle,USA,2006: 19-26 [6] Liu Yiqun,Cen Rongwei,Zhang Min,et al.Automatic Search Engineer Evaluation Based on User Behavior Analysis.Journal of Software,2008,19(11): 3023-3032 (in Chinese) (刘奕群,岑荣伟,张 敏,等.基于用户行为分析的搜索引擎自动性能评价.软件学报,2008,19(11): 3023-3032) [7] Sharma H,Jansen B J.Automated Evaluation of Search Engine Performance via Implicit User Feedback // Proc of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.Salvador,Brazil,2005: 649-650 [8] Huffman S B,Hochster M.How Well Does Result Relevance Predict Session Satisfaction? // Proc of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.Amsterdam,Netherlands,2007: 567-574 [9] Hassan A,Jones R,Klinkner K.Beyond DCG: User Behavior as a Predictor of a Successful Search // Proc of the 3rd International Conference on Web Search and Data Mining,Ann Arbor,USA,2010: 221-230 [10] Carletta J.Assessing Agreement on Classification Tasks: The Kappa Statistic.Computational Linguistics,1996,22(2): 249-254
2012, Vol. 25 
