Knowledge Base Question Answering Method Incorporating Fact Text
WANG Guangxiang1,2, HE Shizhu1,2, LIU Kang1,2, YU Zhengtao1,2, GAO Shengxiang1,2, GUO Junjun1,2
1. Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650504 2. Yunnan Key Laboratory of Artificial Intelligence, Kunming Uni-versity of Science and Technology, Kunming 650500 3. National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190
Abstract In natural language problems, the relationship expression in the knowledge base is diversified. Therefore, matching the answers of the knowledge base question and answer through representation learning is still a challenge. To make up the shortcomings, a knowledge base question answering method incorporating fact text is proposed. Entities, entity types and relationships in the knowledge base are converted into fact text. A pre-trained language model(BERT) is employed for representation. The vector of question and answers in low dimensional semantic space is obtained using the rich semantic mode of BERT. The answer with the closest semantic similarity to the question is matched by calculation. Experiments show that the proposed method is effective and robust in answering common simple questions.
Fund:National Key Research and Development Program of China(No.2018YFC0830101,2018YFC0830105,2018YFC0830100), National Natural Science Foundation of China(No.61533018,61972186,61762056,61472168,61702512), Yunnan High-tech Industry Development Project(No.201606)
Corresponding Authors:
YU Zhengtao, Ph.D., professor. His research interests include natural language processing, information retrieval and machine translation.
About author:: WANG Guangxiang, master student. His research interests include natural language pro-cessing.HE Shizhu, Ph.D., associate professor. His research interests include knowledge graph and natural language processing.LIU Kang, Ph.D., professor. His research interests include knowledge graph and natural language processing.GAO Shengxiang, Ph.D., associate professor. Her research interests include natural language processing.GUO Junjun, Ph.D., associate professor. His research interests include natural language processing.
[1] BOLLACKER K, EVANS C, PARITOSH P, et al. Freebase: A Collaboratively Created Graph Database for Structuring Human Know-ledge // Proc of the ACM SIGMOD International Conference on Management of Data. New York, USA: ACM, 2008: 1247-1250. [2] LEHMANN J, ISELE R, JAKOB M, et al. DBpedia-A Large-Scale, Multilingual Knowledge Base Extracted from Wikipedia. Semantic Web, 2015, 6(2): 167-195. [3] SUCHANEK F M, KASNECI G, WEIKUM G. YAGO: A Core of Semantic Knowledge // Proc of the 16th International Conference on World Wide Wed. New York, USA: ACM, 2007: 697-706. [4] STEEDMAN M. Surface Structure and Interpretation. Linguistic Inquiry Monographs, 1996, 43(3): 271-276. [5] RENGASWAMY R, WENKATASUBRAMANIAN V. A Syntactic Pattern-Recognition Approach for Process Monitoring and Fault Diagnosis. Engineering Applications of Artificial Intelligence, 1995, 8(1): 35-51. [6] BERANT J, CHOU A, FRASTIG R, et al. Semantic Parsing on Freebase from Question-Answer Pairs // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2013: 1533-1544. [7] KWIATKOWSKI T, ZETTLEMOYER L, GOLDWATER S, et al. Lexical Generalization in CCG Grammar Induction for Semantic Pa-rsing // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2011: 1512-1523. [8] YAO X C, VAN DURME B. Information Extraction over Structured Data: Question Answering with Freebase // Proc of the 52nd Annual Meeting of the Association for Computational Linguistics(Long Papers). Stroudsburg, USA: ACL, 2014: 956-966. [9] BORDES A, CHOPRA S, WESTON J. Question Answering with Subgraph Embeddings // Proc of the Conference on Empirical Me-thods in Natural Language Processing. Stroudsburg, USA: ACL, 2014: 615-620. [10] DONG L, WEI F R, ZHOU M, et al. Question Answering over Freebase with Multi-column Convolutional Neural Networks // Proc of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing(Long Papers). Stroudsburg, USA: ACL, 2015: 260-269. [11] HAO Y C, ZHANG Y Z, LIU K, et al. An End-to-End Model for Question Answering over Knowledge Base with Cross-Attention Combi-ning Global Knowledge // Proc of the 55th Annual Meeting of the Association for Computational Linguistics(Long Papers). Stroudsburg, USA: ACL, 2017: 221-231. [12] YIN W P, YU M, XIANG B, et al. Simple Question Answering by Attentive Convolutional Neural Network[C/OL]. [2020-12-22]. https://arxiv.org/pdf/1606.03391.pdf. [13] GOLUB D, HE X D. Character-Level Question Answering with Attention[C/OL]. [2020-12-22]. https://arxiv.org/pdf/1604.00727v1.pdf. [14] LUKOVNIKOV D, FISCHER A, LEHMANN J, et al. Neural Network-Based Question Answering over Knowledge Graphs on Word and Character Level // Proc of the 26th International Conference on World Wide Web. New York, USA: ACM, 2017: 1211-1220. [15] HUANG X, ZHANG J Y, LI D C, et al. Knowledge Graph Embedding Based Question Answering // Proc of the 12th ACM International Conference on Web Search and Data Mining. New York, USA: ACM, 2019: 105-113. [16] BORDES A, USNIER N, CARCIA-BURAN A, et al. Translating Embeddings for Modeling Multi-relational Data // Proc of the 26th International Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2013, II: 2787-2795. [17] WANG Z, ZHANG J W, FENG J L, et al. Knowledge Graph Embedding by Translating on Hyperplanes // Proc of the 28th AAAI Conference on Artificial Intelligence. Palo Alto, USA; AAAI Press, 2014: 1112-1119. [18] LIN Y K, LIU Z Y, SUN M S, et al. Learning Entity and Relation Embeddings for Knowledge Graph Completion // Proc of the 29th AAAI Conference on Artificial Intelligence. Palo Alto, USA; AAAI Press, 2015: 2181-2187. [19] 俞敬松,魏 一,张永伟.基于BERT的古文断句研究与应用.中文信息学报, 2019, 33(11): 57-63. (YU J S, WEI Y, ZHANG Y W. Automatic Ancient Chinese Texts Segmentation Based on BERT. Journal of Chinese Information Processing, 2019, 33(11): 57-63.) [20] BERANT J, LIANG P. Semantic Parsing via Paraphrasing // Proc of the 52nd Annual Meeting of the Association for Computational Linguistics(Long Papers). Stroudsburg, USA: ACL, 2014: 1415-1425. [21] BORDES A, USUNIER N, CHOPRA S, et al. Large-Scale Simple Question Answering with Memory Networks[C/OL]. [2020-12-22]. https://arxiv.org/pdf/1506.02075.pdf. [22] PENNIGNTON J, SOCHER R, MANNING C D. GloVe: Global Vectors for Word Representation // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2014: 1532-1543. [23] REIMERS N, GUREVYCH I. Sentence-BERT: Sentence Embe-ddings Using Siamese BERT-Networks // Proc of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg, USA: ACL, 2019: 3982-3992. [24] SCHROFF F, KALENICHENKO D, PHILBIN J. FaceNet: A Unified Embedding for Face Recognition and Clustering // Proc of the 28th IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 815-823. [25] MOHAMMED S, SHI P, LIN J. Strong Baselines for Simple Question Answering over Knowledge Graphs with and without Neural Networks[C/OL]. [2020-12-22]. https://arxiv.org/pdf/1712.01969.pdf. [26] KRIZHEVSKY A, SUTSKEVER H, HINTON G E, et al. ImageNet Classification with Deep Convolutional Neural Networks // Proc of the 25th International Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2012, I: 1097-1105. [27] BOWMAN S R, ANGELI G, POTTS C, et al. A Large Annotated Corpus for Learning Natural Language Inference[C/OL]. [2020-12-22]. https://arxiv.org/pdf/1508.05326.pdf. [28] CER D, DIAB M, AGIRRE E, et al. SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation // Proc of the 11th International Workshop on Semantic Evaluations. Berlin, Germany: Springer, 2017: 1-14.
2021, Vol. 34 
