Computerized Adaptive Testing Method Based on Reinforcement Learning for Series Diagnosis
LIU Zirui1, WU Jinze2, YAO Fangzhou1, LIU Qi1, CHEN Enhong1, SHA Jing2, WANG Shijin2, SU Yu3
1. School of Computer Science and Technology, University of Sci-ence and Technology of China, Hefei 230027; 2. iFLYTEK Co., Ltd, Hefei 230088; 3. School of Computer and Artificial Intelligence, Hefei Normal University, Hefei 230061
Abstract：Computerized adaptive testing is designed to select appropriate questions for students based on their historical performance, thereby measuring their actual ability quickly and effectively. However, in intelligent education scenarios, the existing selection strategy of traditional computerized adaptive testing is still faced with some problems such as target complexity and knowledge sparseness. To solve these problems, a computerized adaptive testing method based on reinforcement learning for series diagnosis is proposed in this paper to accurately assess students' knowledge proficiency for intelligent scenarios. A student simulator and a student portrait model based on series diagnosis model are adopted. To address the complexity of computerized adaptive testing goals in real-world scenarios, five evaluation indicators are designed, including accuracy of weak points, coupling of prediction performance, adaptive testing duration, testing anomaly rate and testing difficulty structure. Furthermore, a selection strategy for reinforcement learning based computerized adaptive testing is proposed. The dual-channel self-attention learning module and the contradiction learning module are utilized to ameliorate knowledge sparseness problem. Experiments on real datasets show that the proposed selection strategy not only efficiently measures students' actual abilities, but also optimizes their answering experience. The selected questions exhibit a certain level of interpretability, and the method provides a feasible solution for computerized adaptive testing in intelligent education scenarios.
刘子瑞, 吴金泽, 姚方舟, 刘淇, 陈恩红, 沙晶, 王士进, 苏喻. 面向序列诊断的强化计算机自适应测验方法[J]. 模式识别与人工智能, 2024, 37(1): 13-26.
LIU Zirui, WU Jinze, YAO Fangzhou, LIU Qi, CHEN Enhong, SHA Jing, WANG Shijin, SU Yu. Computerized Adaptive Testing Method Based on Reinforcement Learning for Series Diagnosis. Pattern Recognition and Artificial Intelligence, 2024, 37(1): 13-26.
[1] MAGHSUDI S, LAN A, XU J, et al. Personalized Education in the Artificial Intelligence Era: What to Expect Next. IEEE Signal Pro-cessing Magazine, 2021: 38(3): 37-50. [2] 刘淇,陈恩红,朱天宇,等.面向在线智慧学习的教育数据挖掘技术研究.模式识别与人工智能, 2018, 31(1): 77-90. (LU Q, CHEN E H, ZHU T Y, et al. Research on Educational Data Mining for Online Intelligent Learning. Pattern Recognition and Artificial Intelligence, 2018, 31(1): 77-90.) [3] WAINER H, DORANS N J, EIGNOR D, et al. Computerized Ada-ptive Testing: A Primer(2nd Edition). Quality of Life Research, 2001, 10(8): 733-734. [4] WANG Z, CHEN J L.CAT: A Case Study of GRE. CELEA Jour-nal, 2004, 27(1): 12-17. [5] 朱天宇,黄振亚,陈恩红,等.基于认知诊断的个性化试题推荐方法.计算机学报, 2017, 40(1): 176-191. (ZHU T Y, HUANG Z Y, CHEN E H, et al. Cognitive Diagnosis Based Personalized Question Recommendation. Chinese Journal of Computers, 2017, 40(1): 176-191.) [6] VIE J J, POPINEAU F, BRUILLARD É, et al. A Review of Recent Advances in Adaptive Assessment // PEÑA-AYALA A, ed. Lear-ning Analytics: Fundaments, Applications, and Trends. Berlin, Germany: Springer, 2017: 113-142. [7] SUTTON R S, BARTO A G. Reinforcement Learning: An Introduc-tion. Cambridge, USA: MIT Press, 1998. [8] MNIH V, KAVUKCUOGLU K, SILVER D,et al. Playing Atari with Deep Reinforcement Learning[C/OL]. [2023-08-17]. https://arxiv.org/pdf/1312.5602.pdf. [9] LIU Q, SHEN S H, HUANG Z Y, et al. A Survey of Knowledge Tracing[C/OL].[2023-08-17]. https://arxiv.org/pdf/2105.15106.pdf. [10] CORBETT A T, ANDERSON J R.Knowledge Tracing: Modeling the Acquisition of Procedural Knowledge. User Modeling and User-Adapted Interaction, 1995, 4(4): 253-278. [11] PIECH C, BASSEN J, HUANG J, et al.. Deep Knowledge Tracing // Proc of the 28th International Conference on Neural Information Processing Systems. Cambridge,USA: MIT Press, 2015: 505-513. [12] ZHANG J N, SHI X J, KING I, et al. Dynamic Key-Value Me-mory Networks for Knowledge Tracing // Proc of the 26th Interna-tional Conference on World Wide Web. New York, USA: ACM, 2017: 765-774. [13] NAKAGAWA H, IWASAWA Y, MATSUO Y.Graph-Based Know-ledge Tracing: Modeling Student Proficiency Using Graph Neural Network // Proc of the IEEE/WIC/ACM International Conference on Web Intelligence. Washington, USA: IEEE, 2019: 156-163. [14] TONG S W, LIU Q, HUANG W, et al. Structure-Based Know-ledge Tracing: An Influence Propagation View // Proc of the IEEE International Conference on Data Mining. Washington, USA: IEEE, 2020: 541-550. [15] GHOSH A, HEFFERNAN N, LAN A S.Context-Aware Attentive Knowledge Tracing // Proc of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2020: 2330-2339. [16] PANDEY S, KARYPIS G.A Self-Attentive Model for Knowledge Tracing[C/OL]. [2023-08-17].https://arxiv.org/pdf/1907.06837.pdf. [17] SHEN S H, LIU Q, CHEN E H, et al. Convolutional Knowledge Tracing: Modeling Individualization in Student Learning Process // Proc of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York,USA: ACM, 2020: 1857-1860. [18] 罗照盛. 项目反应理论基础. 北京:北京师范大学出版社, 2012. (LUO Z S. Item Response Theory. Beijing, China: Beijing Nor-mal University Publishing Group, 2012.) [19] LORD F M. Some Test Theory for Tailored Testing[C/OL]. [2023-08-17]. https://onlinelibrary.wiley.com/doi/epdf/10.1002/j.2333-8504.1968.tb00562.x [20] LORD F M.Applications of Item Response Theory to Practical Tes-ting Problems. Hillsdale, USA: Lawrence Erlbaum Associates, 1980. [21] CHANG H H, YING Z L.A Global Information Approach to Computerized Adaptive Testing. Applied Psychological Measurement, 1996, 20(3): 213-229. [22] BI H Y, MA H P, HUANG Z Y, et al. Quality Meets Diversity: A Model-Agnostic Framework for Computerized Adaptive Testing[C/OL].[2023-08-17]. https://arxiv.org/pdf/2101.05986.pdf. [23] GHOSH A, LAN A.BOBCAT: Bilevel Optimization-Based Com-puterized Adaptive Testing // Proc of the 30th International Joint Conference on Artificial Intelligence Main Track. San Francisco, USA: IJCAI, 2021: 2410-2417. [24] ZHUANG Y, LIU Q, HUANG Z Y, et al. Fully Adaptive Frame-work: Neural Computerized Adaptive Testing for Online Education. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(4): 4734-4742. [25] BELLMAN R.A Markovian Decision Process. Journal of Mathe-matics and Mechanics, 1957, 6(5): 679-684. [26] WATKINS C J C H, DAYAN P. Technical Note. Machine Lear-ning, 1992, 8: 279-292. [27] BERNER C, BROCKMAN G, CHAN B, et al. Dota 2 with Large Scale Deep Reinforcement Learning[C/OL].[2023-08-17]. https://arxiv.org/pdf/1912.06680.pdf. [28] SAK H, SENIOR A, BEAUFAYS F.Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition[C/OL]. [2023-08-17].https://arxiv.org/pdf/1402.1128v1.pdf. [29] BAKER R S J D, CORBETT A T, ALEVEN V. More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing // Proc of the Inter-national Conference on Intelligent Tutoring Systems. Berlin, Ger-many: Springer, 2008: 406-415. [30] VASWANI A, SHAZEER N, PARMAR N, et al.Attention Is All You Need // Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2017: 6000-6010. [31] BURNS J E.Glossary of Terms. BJR Supplement, 1996, 25(3): 183-188.
2024, Vol. 37 
