Structure and Performance Analysis of Open Domain QA System
DU Yong-Ping1, HUANG Xuan-Jing2
1.College of Computer Sciences, Beijing University of Technology, Beijing 100124 2.Department of Computer Science and Engineering, Fudan University, Shanghai 200433
Abstract：Open domain question answering (QA) has drawn much attention from the natural language processing communities. A pattern based question answering system is introduced, and the deep performance analysis and the evaluation of the QA system are presented. The impact of parameter tuning and training set size on system performance is discussed as well. Meanwhile, the t-test results denote the significance of the performance improvement by different factors. Natural language processing tools are used in the QA system, such as the syntax parser and the named entity recognition tool. Analysis results indicate that these tools play an important role in the QA system.
[1] Moldvan D I, Rus V. Logic Form Transformation of WordNet and Its Applicability to Question Answering // Proc of the 39th Annual Meeting of the Association for Computational Linguistics. Toulouse, France, 2001: 402-409 [2] Soubbotin M M, Soubbotin S M. Use of Patterns for Detection of Likely Answer Strings: A Systematic Approach // Proc of the 11th Text Retrieval Conference. Gaithersbarg, USA, 2001: 175-182 [3] Cui Huan, Cai Dongfeng, Miao Xuelei. Research on Web-Based Chinese Question Answering System and Answer Extraction. Journal of Chinese Information Processing, 2004, 18(3): 24-31 (in Chinese) (崔 桓,蔡东风,苗雪雷.基于网络的中文问答系统及信息抽取算法研究.中文信息学报, 2004, 18(3): 24-31) [4] Danilo G, Bernardo M, Ido D, et al. The Third PASCAL Recognizing Textual Entailment Challenge // Proc of the Workshop on Textual Entailment and Paraphrasing. Prague, Czech Republic, 2007: 1-9 [5] Breck E, Burger J, Ferro L, et al. How to Evaluate Your Question Answering System Every Day and Still Get Real Work Done // Proc of the 2nd Conference on Language Resources and Evaluation. Athens, Greece, 2000: 1495-1500 [6] Sleator D D K, Temperley D. Parsing English with a Link Grammar. Technical Report, CMU-CS-91-196, Pittsburgh, USA: Carnegie Mellon University. Department of Computer Science, 1991 [7] Du Yongping, Huang Xuanjing, Wu Lide. Effectively Implementing a Pattern Learning Method in the Question Answering System. Journal of Computer Research and Development, 2006, 43(3): 449-455 (in Chinese) (杜永萍,黄萱菁,吴立德.模式学习在QA系统中的有效实现.计算机研究与发展, 2006, 43(3): 449-455) [8] Ravichandran D, Hovy E. Learning Surface Text Patterns for a Question Answering System // Proc of the 40th Annual Meeting of the Association for Computational Linguistics. Philadelphia, USA, 2002: 41-47
2009, Vol. 22 
