|
|
|
| Quantum Interference Based Duet-Feature Text Representation Model |
| GAO Hui1, ZHANG Peng1, ZHANG Jing1 |
| 1. College of Intelligence and Computing, Tianjin University, Tianjin 300350 |
|
|
|
|
Abstract In the field of information retrieval, quantum interference theory is applied to the study of core issues such as document relevance and order effects, aiming at modeling quantum-like interference phenomena caused by user cognition. Based on the language understanding task, the mathematical tools of quantum theory are utilized to analyze the semantic evolution phenomenon in the semantic combination process. A quantum interference based duet-feature text representation model(QDTM) is proposed. The reduced density matrix is taken as the core component of language representation to effectively model semantic interference information at the dimension-level. On this basis, a model structure is constructed to capture global and local feature information, meeting the semantic feature requirements of different granularities in the language understanding process. Experiments on text classification datasets and question and answering datasets show that QDTM outperforms quantum-inspired language models and neural network text matching models.
|
|
Received: 16 October 2023
|
|
|
| Fund:National Natural Science Foundation of China(No.62276188,61876129), Tianjin University-Wenge Joint Laboratory Project |
|
Corresponding Authors:
ZHANG Peng, Ph.D., professor. His research interests include information retrieval, natural language processing, deep learning and theoretical research on quantum cognition.
|
| About author:: GAO Hui, Ph.D. candidate. Her research interests include quantum semantic understanding and evaluation of large language mo-dels. ZHANG Jing, Ph.D. candidate. Her research interests include large model compre-ssion and acceleration, and few-shot learning based on large models. |
|
|
|
[1] NGUYEN T, ROSENBERG M, SONG X, et al. MS MARCO: A Human Generated Machine Reading Comprehension Dataset[C/OL].[2023-10-05]. https://www.ceur-ws.org/Vol-1773/CoCoNIPS_2016_paper9.pdf.
[2] DONG G T, LI R M, WANG S R, et al. Bridging the KB-Text Gap: Leveraging Structured Knowledge-Aware Pre-training for KBQA//Proc of the 32nd ACM International Conference on Information and Knowledge Management. New York, USA: ACM, 2023: 3854-3859.
[3] VASWANI A, SHAZEER N, PARMAR N, et al. Attention Is All You Need//Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2017: 6000-6010.
[4] HOU Y X, SONG D W. Characterizing Pure High-Order Entanglements in Lexical Semantic Spaces via Information Geometry//Proc of the 3rd International Symposium on Quantum Interaction. Berlin, Germany: Springer, 2009: 237-250.
[5] WANG B Y, PENG Z, LI J F, et al. Exploration of Quantum Interference in Document Relevance Judgement Discrepancy. Entropy, 2016, 18(4). DOI: 10.3390/e18040144.
[6] HOU Y X, ZHAO X Z, SONG D W, et al. Mining Pure High-Order Word Associations via Information Geometry for Information Retrie-val. ACM Transactions on Information Systems, 2013: 31(3). DOI: 10.1145/2493175.2493177.
[7] SORDONI A, NIE J Y, BENGIO Y. Modeling Term Dependencies with Quantum Language Models for IR//Proc of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM, 2013: 653-662.
[8] ZHANG P, NIU J B, SU Z, et al. End-to-End Quantum-Like Language Models with Application to Question Answering//Proc of the 32nd AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence. Palo Alto, USA: AAAI Press, 2018: 5666-5673.
[9] LI Q C, MELUCCI M, TIWARI P. Quantum Language Model-Based Query Expansion//Proc of the ACM SIGIR International Confe-rence on Theory of Information Retrieval. New York, USA: ACM, 2018: 183-186.
[10] GAO H, ZHANG P.A Neural Matching Model Based on Quantum Interference and Quantum Many-Body System[C/OL].[2023-10-05].https://tensorworkshop.github.io/NeurIPS2020/accepted_papers/Neurips_2020_workshop.pdf.
[11] GUO P, WANG P P.QHAN: Quantum-Inspired Hierarchical Attention Mechanism Network for Question Answering. International Journal on Artificial Intelligence Tools, 2023, 32(5). DOI: 10.1142/S0218213023600096.
[12] JIANG Y Y, ZHANG P, GAO H, et al. A Quantum Interference Inspired Neural Matching Model for AD-HOC Retrieval//Proc of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM, 2020: 19-28.
[13] ZHANG P, HUI W J, WANG B Y, et al. Complex-Valued Neural Network-based Quantum Language Models. ACM Transactions on Information Systems, 2022, 40(4). DOI: 10.1145/3505138.
[14] ZHANG W T, GAN G B, GAO H, et al. A Measurement-Based Quantum-Like Language Model for Text Matching//Proc of the International Conference on Neural Information Processing. Berlin, Germany: Springer, 2022, III: 38-47.
[15] LI S Z, ZHANG P, GAN G B, et al. Hypoformer: Hybrid Decomposition Transformer for Edge-Friendly Neural Machine Translation//Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2022: 7056-7068.
[16] UPRETY S, GKOUMAS D, SONG D W.A Survey of Quantum Theory Inspired Approaches to Information Retrieval. ACM Computing Surveys, 2020, 53(5). DOI: 10.1145/3402179.
[17] ZHANG P, GAO H, ZHANG J, et al. Quantum-Inspired Neural Language Representation, Matching and Understanding. Foundations and Trends® in Information Retrieval, 2023, 16(4/5): 318-509.
[18] LIU Y C, LI Q C, WANG B Y, et al. A Survey of Quantum-Cognitively Inspired Sentiment Analysis Models. ACM Computing Surveys, 2024, 56(1). DOI: 10.1145/3604550.
[19] PANG B, LEE L, VAITHYANATHAN S.Thumbs up? Sentiment Classification Using Machine Learning Techniques//Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2002: 79-86.
[20] HU M Q, LIU B. Mining and Summarizing Customer Reviews//Proc of the 10th ACM SIGKDD International Conference on Know-ledge Discovery and Data Mining. New York, USA: ACM, 2004: 168-177.
[21] WIEBE J, WILSON T, CARDIE C. Annotating Expressions of Opinions and Emotions in Language. Language Resources and Evaluation, 2005, 39(2): 165-210.
[22] SOCHER R, PERELYGIN A, WU J Y, et al. Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank//Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2013: 1631-1642.
[23] LI X, ROTH D.Learning Question Classifiers: The Role of Semantic Information. Natural Language Engineering, 2006, 12(3): 229-249.
[24] YANG Y, YIH S W, MEEK C. WIKIQA: A Challenge Dataset for Open-Domain Question Answering//Proc of the Conference on Em-pirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2015: 2013-2018.
[25] WANG M Q, SMITH N A, MITAMURA T. What Is the Jeopardy Model? A Quasi-Synchronous Grammar for QA//Proc of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Stroudsburg, USA: ACL, 2007: 22-32.
[26] KIM Y. Convolutional Neural Networks for Sentence Classification//Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2014: 1746-1751.
[27] CHEN Y W, PAN Y, DONG D Y.Quantum Language Model with Entanglement Embedding for Question Answering. IEEE Transactions on Cybernetics, 2023, 53(6): 3467-3478.
[28] LEI Y, HERMANN K M, BLUNSOM P, et al. Deep Learning for Answer Sentence Selection[C/OL].[2023-10-05]. https://arxiv.org/pdf/1412.1632.pdf.
[29] DI W, NYBERG E. A Long Short-Term Memory Model for Answer Sentence Selection in Question Answering//Proc of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Proce-ssing(Short Papers). Stroudsburg, USA: ACL, 2015: 707-712. |
|
|
|
2024, Vol. 37 
