基于统计语言模型的蒙古文词切分<sup>*</sup>

摘要
图/表
参考文献
相关文章 (2)

全文: PDF (320 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要通过对蒙古文词切分技术的分析，利用规则作为切分的基础，提出一种统计和规则相结合的蒙古文词切分方法.这种方法利用蒙古语统计语言模型作为排歧依据，使用的语言模型有基于词性的语言模型和Skip-N语言模型.其词切分准确率比基于规则的系统有较大提高.

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	侯宏旭
	刘群
	那顺乌日图
	牧仁高娃
	李锦涛

关键词 ：蒙古语, 词切分, 语言模型, 词干词缀

Abstract：Based on the analysis of Mongolian segmentation technique and the rules used as the foundation of word segmentation, a hybrid word segmentation method is proposed. It uses Mongolian statistical language model to eliminate the ambiguity in Mongolian word segmentation. A POS language model and a Skip-N language model are used, and an experiment system is thus created. The experimental results are better than those of the system based on rules.

Key words： Mongolian Word Segmentation Language Model Etyma and Suffix

收稿日期: 2008-03-03

ZTFLH:

TP391

基金资助:内蒙古自然科学基金项目(No.200607010805)、国家973计划前期研究项目(No.2007CB316503)资助

作者简介: 侯宏旭，男，1972年生，副教授，主要研究方向为中文信息处理.E-mail: cshhx@imu.edu.cn.刘群，男，1966年生，研究员，主要研究方向为自然语言处理.那顺乌日图，男，1959年生，教授，主要研究方向为计算语言学.牧仁高娃，女，1982年生，硕士研究生，主要研究方向为计算语言学.李锦涛，男，1962年生，研究员，主要研究方向为数字媒体处理技术.

引用本文:

侯宏旭，刘群，那顺乌日图，牧仁高娃，李锦涛. 基于统计语言模型的蒙古文词切分^*[J]. 模式识别与人工智能, 2009, 22(1): 108-112. HOU Hong-Xu, LIU Qun, Nasanurtu, Murengaowa, LI Jin-Tao. Mongolian Word Segmentation Based on Statistical Language Model. , 2009, 22(1): 108-112.

链接本文:

http://manu46.magtech.com.cn/Jweb_prai/CN/ 或 http://manu46.magtech.com.cn/Jweb_prai/CN/Y2009/V22/I1/108

[1] Nasanurtu. A Segmentation System of Mongolian Etyma, Stem and Affix. Journal of Inner Mongolia University: Humanities and Social Sciences, 1997, 29(2): 53-57 (in Chinese)
(那顺乌日图.蒙古文词根、词干、词尾自动切分系统.内蒙古大学学报:人文社会科学版, 1997, 29(2): 53-57)
[2] Hua Shabao. The POS Tagger System for Mongolian Corpus. Journal of Inner Mongolia University: Humanities and Social Sciences, 1999, 31(5): 33-37 (in Chinese)
(华沙宝.对蒙古文语料库的词类标注系统——AYIMAG.内蒙古大学学报: 人文社会科学版, 1999, 31(5): 33-37)
[3] Hou Hongxu, Liu Qun, Zhang Yujie, et al. Research and Implement of the 2005 HTRDP(863) Evaluation on Machine Translation. Journal of Chinese Information Processing, 2006, 20(Z1): 7-18 (in Chinese)
(侯宏旭,刘群,张玉洁,等.2005年度863机器翻译评测方法研究与实施.中文信息学报, 2006, 20(Z1): 7-18)
[4] Badma-Odsar. A Study of Part of Speech Classification of Mongolian Language. Journal of the Central University for Nationalities: Philosophy and Social Sciences Edition, 2004, 31(3): 94-100 (in Chinese)
(巴达玛敖德斯尔.面向信息处理的蒙古语词语分类体系研究.中央民族大学学报:哲学社会科学版, 2004, 31(3): 94-100)
[5] Nasanurtu. Semantic Research for the Mongolian Language to Be Oriented to Information Processing. Journal of Inner Mongolia University: Humanities and Social Sciences, 2002, 34(5): 43-48 (in Chinese)
(那顺乌日图.关于面向信息处理的蒙古语语义研究.内蒙古大学学报:人文社会科学版, 2002, 34(5): 43-48)
[6] Hua Shabao. The Technological Countermeasure to Deal with the Net Information in Mongolian. Minority Languages of China, 2002, 6: 58-60 (in Chinese)
(华沙宝.蒙古文网络信息技术处理的对策.民族语文, 2002, 6: 58-60)
[7] Hou Hongxu, Deng Dan, Zou Gang, et al. An EBMT System Based on Word Alignment // Proc of the 4th International Workshop of Spoken Language Translation. Trento, Italy, 2004: 47-49
[8] Zhang Huaping, Yu Hongkui, Xiong Deyi, et al. HHMM-Based Chinese Lexical Analyzer ICTCLAS // Proc of the 2nd SIGHAN Workshop on Chinese Language Processing. Sapporo, Japan, 2003: 184-187
[9] Ye Jiaming. Research and Implement of Mongolian Lexical Analysis Based on Rules. Master Dissertation. Beijing, China: Peking University. School of Electronics Engineering and Computer Science, 2005 (in Chinese)
(叶嘉明.基于规则的蒙古语词法分析研究与实现.硕士学位论文.北京:北京大学.信息科学技术学院, 2005)
[10] Liu Qun, Zhan Weidong, Chang Baobao, et al. Computing Model and Language Model of Chinese-English Translation System // Proc of the 3rd Intelligent Interface and Intelligent Application. Zhangjiajie, China, 1997: 253-258 (in Chinese)
(刘群,詹卫东,常宝宝,等.一个汉英机器翻译系统的计算模型与语言模型//第3届全国智能接口与智能应用学术会议.张家界, 1997: 253-258)
[11]Hou Hongxu, Liu Qun, Liu Zhiwen. Skip-N Mongolian Statistical Language Model. Journal of Inner Mongolia University: Natural Sciences, 2008, 39(2): 220-224 (in Chinese)
(侯宏旭,刘群,刘志文.Skip-N蒙古文统计语言模型.内蒙古大学学报:自然科学版, 2008, 39(2): 220-224)
[12] Katz S M. Estimation of Probabilities from Sparse Data for the Language Model Component of a Speech Recognizer. IEEE Trans on Acoustics, Speech and Signal Processing, 1987, 35(3): 400-401
[13] Och F J. Minimum Error Rate Training in Statistical Machine Translation // Proc of the 41st Annual Meeting on Association for Computational Linguistics. Sapporo, Japan, 2003: 160-167