Chinese Multi-paragraph Reading Comprehension Model
ZHAO Junyao1, 2, PANG Liang1, SU Lixin1, LAN Yanyan1, GUO Jiafeng1, CHENG Xueqi1
1.Key Laboratory of Network Data Science and Technology, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190;
2.School of Computer and Control Engineering, University of Chinese Academy of Sciences, Beijing 100190
In the Chinese multi-paragraph reading comprehension task, three properties should be taken into account: the sparsity of evidence paragraph, the diversity of Chinese semantic and the validity of answer snippet. To solve these problems, a Chinese multi-paragraph reading comprehension model, CMPReader, is proposed. In CMReader, data augmentation is exploited to learn the paragraphs with no answer. Word level encoding and Chinese word tag are added to enrich the Chinese semantic representation, and the features of answer snippet are employed by the answer verifier model to choose the right answer. CMPReader is applied to the CIPS-SOGOU factoid question answer dataset, and the results show that the average of exact match score and F1 score are increased.
Supported by National Key R&D Program of China(No.2016QY02D0405), National Natural Science Foundation of China(No.61425016,61472401,61722211,61872338,61773362,20180290), Youth Innovation Promotion Association CAS(No.20144310,2016102),
About author:: (ZHAO Junyao, master student. His research interests include natural language processing and question answering system.)(PANG Liang(Corresponding author), Ph.D., assistant researcher. His research interests include natural language processing and machine learning.)(SU Lixin, Ph.D. candidate. His research interests include information retrieval and question answering system.)(LAN Yanyan, Ph.D., associate professor. Her research interests include machine lear-ning and data mining.)(GUO Jiafeng, Ph.D., professor. His research interests include data mining and information retrieval.)(CHENG Xueqi, Ph.D., professor. His research interests include network science and social computing, web search and data mi-ning.)
[1] BERANT J, SRIKUMAR V, CHEN P C, et al. Modeling Biological Processes for Reading Comprehension // Proc of the Conference on Empirical Methods in Natural Language Processing. Berlin, Germany: Springer, 2014: 1499-1510.
[2] DUNN M, SAGUN L, HIGGINS M, et al. SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine[J/OL].[2018-09-15]. https://arxiv.org/pdf/1704.05179.pdf.
[3] WU F, LAO N, BLITZER J, et al. Fast Reading Comprehension with ConvNets[J/OL].[2018-09-15]. https://arxiv.org/pdf/1711.04352.pdf.
[4] YU A W, DOHAN D, LUONG M T, et al. QANet: Combining Local Convolution with Global Self-attention for Reading Comprehension[J/OL].[2018-09-15]. https://arxiv.org/pdf/1804.09541.pdf.
[5] HERMANN K M, KŎCISKÝ T, GREFENSTETTE E, et al. Teaching Machines to Read and Comprehend // CORTES C, LAWRENCE N D, LEE D D, et al., eds. Advances in Neural Information Processing Systems 28. Cambridge, USA: The MIT Press, 2015: 1693-1701.
[6] HILL F, BORDES A, CHOPRA S, et al. The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations[J/OL].[2018-09-15]. https://arxiv.org/pdf/1511.02301.pdf.
[7] CHEN D Q, FISCH A, WESTON J, et al. Reading Wikipedia to Answer Open-Domain Questions[J/OL].[2018-09-15]. https://arxiv.org/pdf/1704.00051.pdf.
[8] WANG S H, JIANG J.Machine Comprehension Using Match-LSTM and Answer Pointer[J/OL]. [2018-09-15].https://arxiv.org/pdf/1608.07905.pdf.
[9] VINYALS O, FORTUNATO M, JAITLY N.Pointer Networks // CORTES C, LAWRENCE N D, LEE D D, et al., eds. Advances in Neural Information Processing Systems 28. Cambridge, USA: The MIT Press, 2015: 2692-2700.
[10] SEO M J, KEMBHAVI A, HAJISHIRZI H, et al. Bidirectional Attention Flow for Machine Comprehension[J/OL].[2018-09-15]. https://arxiv.org/pdf/1611.01603.pdf.
[11] SRIVASTAVA R K, GREFF K, SCHMIDHUBER J.Highway Networks[J/OL]. [2018-09-15].https://arxiv.org/pdf/1505.00387.pdf.
[12] HILL F, CHO K, KORHONEN A.Learning Distributed Representations of Sentences from Unlabelled Data[C/OL]. [2018-09-15].https://arxiv.org/pdf/1602.03483v1.pdf.
[13] FRIEDMAN J H. Stochastic Gradient Boosting. Computational Statistics and Data Analysis, 2002, 38(4): 367-378.
[14] KINGMA D P, BA J L. Adam: A Method for Stochastic Optimization[J/OL]. [2018-09-15]. https://arxiv.org/pdf/1412.6980.pdf.
[15] RAIPURKAR P, ZHANG J, LOPYREV K, et al. Squad: 100,000+ Questions for Machine Comprehension of Text[J/OL].[2018-09-15]. https://arxiv.org/pdf/1606.05250.pdf.
2019, Vol. 32 
