Dynamic Multiple-level Semantic Extraction Model Based on External Knowledge
JIANG Wenchao1,2, ZHUANG Zhigang1, TU Xuping2, LI Chuanjie3, LIU Haibo1,
1.School of Computers, Guangdong University of Technology,Guangzhou 510006; 2.Guangdong Electronics Industry Institute, Dongguan 523808; 3.Guangdong Guangxin Communications Services Company Ltd., Guangzhou 510630
Abstract To resolve the problems of semantic understanding and answer extraction in complex multiple context machine reading comprehension environments, a dynamic multiple-level semantic extraction model based on external knowledge is presented. Firstly, the optimized gated single cyclic neural network model is utilized to match the text as well as the problem set. Then, the dynamic multiple-dimension bidirectional attention mechanism analysis is implemented on the text and the problem set respectively to improve the semantic matching precision. Next, a dynamic pointer network is utilized to determine the rank of the answers to the questions. Finally, the candidate answers are sorted based on external knowledge and experiences, and the precision of the final answer is improved further. The experimental results show that problem-answer matching accuracy of the proposed model is significantly improved compared with the mainstream models. Furthermore, the proposed model obtains higher robustness in complex reading comprehension tasks in different application scenes.
Fund:Supported by National Natural Science Foundation of China(No.11601006), Natural Science Foundation of Guangdong Province(No.2018A030313061), Guangdong Science and Technology Plan(No.2017B030305003,2017B010124001, 2017B09090100 5), Collaborative Innovation and Platform Environment Construction Project of Guangdong Province(No.2017A070712016)
Corresponding Authors:
(JIANG Wenchao(corresponding author), Ph.D., lecturer. His research interests include cloud computing, complex network and data mining)
About author:: (ZHUANG Zhigang, master student. His research interests include computer vision, image recognition and data mining.)(TU Xuping, Ph.D., associate researcher. His research interests include cloud computing, distributed system and big data analysis.)(LI Chuanjie, master, senior engineer. His research interests include computer vision, image recognition and data mining.)(LIU Haibo, master student. His research interests include computer vision, image re-cognition and data mining.)
[1] HERMANN K M, KOCISKY T, GREFENSTETTE E, et al. Tea-ching Machines to Read and Comprehend // CORTES C, LAWRENCE N D, LEE D D, et al., eds. Advances in Neural Information Processing Systems 28. Cambridge, USA: The MIT Press, 2015: 1693-1701. [2] HOCHREITER S, SCHMIDHUBER J. Long Short-Term Memory. Neural Computation, 1997, 9(8): 1735-1780. [3] HILL F, BORDES A, CHOPRA S, et al. The Goldilocks Principle: Reading Children′s Books with Explicit Memory Representations[C/OL]. [2018-08-20]. https://arxiv.org/pdf/1511.02301.pdf. [4] WESTON J, CHOPRA S, BORDES A. Memory Networks[C/OL]. [2018-08-20]. https://arxiv.org/pdf/1410.3916.pdf. [5] RAJPURKAR P, ZHANG J, LOPYREV K, et al. Squad:100,000+ Questions for Machine Comprehension of Text // Proc of the Confe-rence on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2016: 2383-2392. [6] NGUYEN T, ROSENBERG M, SONG X, et al. MS MARCO: A Human Generated Machine Reading Comprehension Dataset[C/OL]. [2018-08-20]. https://arxiv.org/pdf/1611.09268.pdf. [7] HE W, LIU K, LIU J, et al. DuReader: A Chinese Machine Rea-ding Comprehension Dataset from Real-World Applications[C/OL]. [2018-08-20]. https://arxiv.org/pdf/1711.05073.pdf. [8] WANG S H, JIANG J. Machine Comprehension Using Match-LSTM and Answer Pointer[C/OL]. [2018-08-20]. https://arxiv.org/pdf/1608.07905.pdf. [9] SEO M, KEMBHAVI A, FARHADI A, et al. Bidirectional Attention Flow for Machine Comprehension[C/OL]. [2018-08-20]. https://arxiv.org/pdf/1611.01603.pdf. [10] XIONG C M, ZHONG V, SOCHER R. Dynamic Coattention Networks for Question Answering[C/OL]. [2018-08-20]. https://arxiv.org/pdf/1611.01604.pdf. [11] DUNN M, SAGUN L, HIGGINS M, et al. SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine[C/OL]. [2018-08-20]. https://arxiv.org/pdf/1704.05179.pdf. [12] YU Y, ZHANG W, HASAN K, et al. End-to-End Reading Com-prehension with Dynamic Answer Chunk Ranking[C/OL]. [2018-08-20]. https://arxiv.org/pdf/1610.09996v2.pdf. [13] LEE K, KWIATKOWSKI T, PARIKH A, et al. Learning Recu-rrent Span Representations for Extractive Question Answering[C/OL]. [2018-08-20]. https://arxiv.org/pdf/1611.01436.pdf. [14] WANG S H, YU M, JIANG J, et al. Evidence Aggregation for Answer Re-ranking in Open-Domain Question Answering[C/OL]. [2018-08-20]. https://arxiv.org/pdf/1711.05116.pdf. [15] LONG T, BENGIO E, LOWE R, et al. World Knowledge for Reading Comprehension: Rare Entity Prediction with Hierarchical LSTMs Using External Descriptions // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2017: 825-834. [16] CHO K, VAN MERRIENBOER B, GULCEHRE C, et al. Lea-rning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2014: 1724-1734. [17] LIN C Y. ROUGE: A Package for Automatic Evaluation of Summaries. Text Summarization Branches Out[C/OL]. [2018-08-20]. https://www.aclweb.org/anthology/W04-1013. [18] PAPINENI K, ROUKOS S, WARD T, et al. BLEU: A Method for Automatic Evaluation of Machine Translation // Proc of the 40th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2002: 311-318. [19] KINGMA D P, BA J. Adam: A Method for Stochastic Optimization[C/OL]. [2018-08-20]. https://arxiv.org/pdf/1412.6980.pdf. [20] SHEN Y L, HUANG P S, GAO J F, et al. Reasonet: Learning to Stop Reading in Machine Comprehension // Proc of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2017: 1047-1055. [21] WEISSENBORN D, WIESE G, SEIFFE L. Making Neural QA as Simple as Possible But Not Simpler // Proc of the 21st Conference on Computational Natural Language Learning. Stroudsburg, USA: ACL, 2017: 271-280. [22] WANG W H, YANG N, WEI F R, et al. Gated Self-matching Networks for Reading Comprehension and Question Answering // Proc of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2017, I: 189-198. [23] TAN C Q, WEI F R, YANG N, et al. S-NET: From Answer Extraction to Answer Generation for Machine Reading Comprehension[C/OL]. [2018-08-20]. https://arxiv.org/pdf/1706.04815.pdf. [24] WANG Y Z, LIU K, LIU J, et al. Multi-passage Machine Reading Comprehension with Cross-Passage Answer Verification // Proc of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2018: 1918-1927. [25] HE K M, ZHANG X Y, REN S Q, et al. Deep Residual Learning for Image Recognition[C/OL]. [2018-08-20]. https://arxiv.org/pdf/1512.03385.pdf.
2019, Vol. 32 
