动态融入<em>k</em>近邻知识的领域机器翻译方法

doi:10.16451/j.cnki.issn1003-6059.202411005

Abstract
Figure/Table
References
Related Citation (12)

Download: PDF (909 KB) HTML (1 KB)
Export: BibTeX | EndNote (RIS)

Abstract Domain machine translation methods based on k-nearest neighbour retrieval improve translation quality by incorporating translation knowledge retrieved from a translation knowledge base. Existing methods enhance translation performance by fusing the decoder prediction distribution with k-nearest neighbour knowledge. However, the inaccuracy of the retrieved k-nearest neighbor knowledge may interfere with the prediction results of the model. To address this issue, a domain machine translation method with dynamic incorporation of k-nearest neighbor knowledge is proposed. The confidence of the decoder output distribution is first assessed. With the combination of gating mechanism, the proposed method dynamically decides whether to incorporate the k-nearest-neighbor retrieval results, thereby adjusting the degree of incorporation of k-nearest neighbor knowledge flexibly. The adaptive k-value module is introduced to reduce the interference caused by incorrect k-nearest neighbor knowledge. Besides, the distribution-guided loss is designed to steer the model output approach the target distribution gradually. On four domain-specific German-English machine translation datasets, the proposed method achieves improvements.

Key words： Domain Translation k-Nearest Neighbor Knowledge Confidence Dynamic Incorporation

Received: 12 October 2024

ZTFLH:

TP 391

Fund:Supported by National Natural Science Foundation of China(No.62366027,62166023,62266027), Major Basic Research Project of Yunnan Province(No.202401BC070021), Science and Tech-nology Major Project of Yunnan Province(No.202303AP140008,202302AD080003), ″Double First-Class″ Science and Techno-logy Project of Kunming University of Science and Technology(No.202402AG050007).

Corresponding Authors: LAI Hua, Master. associate professor. His research interests include intelligent information processing.

About author:: HUANG Yuxin, Ph.D., associate profe-ssor. His research interests include natural language processing and text summarization.SHEN Tao, Master student. His research interests include natural language processing and machine translation.JIANG Shuting, Ph.D. candidate. Her re-search interests include natural language processing and neural machine translation.ZENG Hao, Master student. His research interests include natural language processing and machine translation.

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	HUANG Yuxin
	SHEN Tao
	JIANG Shuting
	ZENG Hao
	LAI Hua

Cite this article:

HUANG Yuxin,SHEN Tao,JIANG Shuting等. Domain Machine Translation Method with Dynamic Incorporation of k-Nearest Neighbor Knowledge[J]. Pattern Recognition and Artificial Intelligence, 2024, 37(11): 999-1009.

URL:

http://manu46.magtech.com.cn/Jweb_prai/EN/10.16451/j.cnki.issn1003-6059.202411005 OR http://manu46.magtech.com.cn/Jweb_prai/EN/Y2024/V37/I11/999

[1] SAUNDERS D. Domain Adaptation and Multi-domain Adaptation for Neural Machine Translation: A Survey. Journal of Artificial Intelligence Research, 2022, 75: 351-424.
[2] REDKO I, MORVANT E, HABRARD A, et al. A Survey on Domain Adaptation Theory: Learning Bounds and Theoretical Guarantees[C/OL].[2024-09-10]. https://arxiv.org/pdf/2004.11829.
[3] CHU C H, WANG R. A Survey of Domain Adaptation for Neural Machine Translation // Proc of the 27th International Conference on Computational Linguistics. Stroudsburg, USA: ACL, 2018: 1304-1319.
[4] ZARDINI E, BLANZIERI E, PASTORELLO D. A Quantum k-Nearest Neighbors Algorithm Based on the Euclidean Distance Estimation. Quantum Machine Intelligence, 2024, 6(1). DOI: 10.1007/s42484-024-00155-2.
[5] 张清华,肖嘉瑜,艾志华,等.自适应半径选择的近邻邻域分类器.模式识别与人工智能, 2022, 35(11): 989-998.
(ZHANG Q H, XIAO J Y, AI Z H. Near Neighborhood Classifier with Adaptive Radius Selection. Pattern Recognition and Artificial Intelligence, 2022, 35(11): 989-998.)
[6] SAKAMOTO T, AIZAWA A. Predicting Numerals in Text Using Nearest Neighbor Language Models // Proc of the Findings of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2023: 4795-4809.
[7] KHANDELWAL U, FAN A, JURAFSKY D, et al. Nearest Neighbor Machine Translation[C/OL].[2024-09-10]. https://arxiv.org/pdf/2010.00710.
[8] ZHENG X, ZHANG Z R, GUO J L, et al. Adaptive Nearest Neighbor Machine Translation // Proc of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing(Short Papers). Stroudsburg, USA: ACL, 2021: 368-374.
[9] JIANG H, LU Z Y, MENG F D, et al. Towards Robust k-Nearest-Neighbor Machine Translation // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2022: 5468-5477.
[10] STAP D, MONZ C. Multilingual k-Nearest-Neighbor Machine Trans-lation // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2023: 9200-9208.
[11] NISHIDA Y, MORISHITA M, KAMIGAITO H, et al. Generating Diverse Translation with Perturbed kNN-MT // Proc of the 18th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop. Stroudsburg, USA: ACL, 2024: 9-31.
[12] GAO Y, CAO Z W, MIAO Z J, et al. Efficient k-Nearest-Neighbor Machine Translation with Dynamic Retrieval // Proc of the Findings of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2024: 7990-8001.
[13] YANG Z X, SUN R L, WAN X J. Nearest Neighbor Knowledge Distillation for Neural Machine Translation // Proc of the Confe-rence of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, USA: ACL, 2022: 5546-5556.
[14] ZHU W H, XU J J, HUANG S J, et al. INK: Injecting KNN Knowledge in Nearest Neighbor Machine Translation // Proc of the 61st Annual Meeting of the Association for Computational Linguistics(Long Papers). Stroudsburg, USA: ACL, 2023: 15948-15959.
[15] WANG J Y, WANG K, ZHANG Y Q, et al. Non-parametric, Nearest-Neighbor-Assisted Fine-Tuning for Neural Machine Translation[C/OL].[2024-09-10]. https://arxiv.org/pdf/2305.13648.
[16] ZHU W H, HUANG S J, LÜ Y Z, et al. What Knowledge Is Needed? Towards Explainable Memory for kNN-MT Domain Adaptation // Proc of the Findings of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2023: 2824-2836.
[17] GAO R Z, ZHANG Z R, DU Y C, et al. Nearest Neighbor Machine Translation Is Meta-Optimizer on Output Projection Layer // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2023: 15592-15608.
[18] MENG Y X, LI X Y, ZHENG X Y, et al. Fast Nearest Neighbor Machine Translation // Proc of the Findings of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2022: 555-565.
[19] DEGUCHI H, WATANABE T, MATSUI Y, et al. Subset Retrie-val Nearest Neighbor Machine Translation // Proc of the 61st Annual Meeting of the Association for Computational Linguistics(Long Papers). Stroudsburg, USA: ACL, 2023: 174-189.
[20] ZHENG X, ZHANG Z R, HUANG S J, et al. Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translation // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2021: 4234-4241.
[21] MARTINS P, MARINHO Z, MARTINS A. Efficient Machine Trans-lation Domain Adaptation // Proc of the 1st Workshop on Semiparametric Methods in NLP: Decoupling Logic from Knowledge. Stroudsburg, USA: ACL, 2022: 23-29.
[22] SHI X Y, LIANG Y L, XU J N, et al. Towards Faster k-Nearest-Neighbor Machine Translation[C/OL].[2024-09-10]. https://arxiv.org/pdf/2312.07419.
[23] WANG D X, FAN K, CHEN B X, et al. Efficient Cluster-Based k-Nearest-Neighbor Machine Translation // Proc of the 60th Annual Meeting of the Association for Computational Linguistics(Long Papers). Stroudsburg, USA: ACL, 2022: 2175-2187.
[24] MARTINS P H, ALVES J, VAZ T, et al. Empirical Assessment of kNN-MT for Real-World Translation Scenarios // Proc of the 24th Annual Conference of the European Association for Machine Translation. Stroudsburg, USA: ACL, 2023: 115-124.
[25] ZHU W H, ZHAO Q F, LÜ Y Z, et al. kNN-BOX: A Unified Framework for Nearest Neighbor Generation // Proc of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations. Stroudsburg, USA: ACL, 2024: 10-17.
[26] WANG Q, WENG R X, CHEN M. Learning Decoupled Retrieval Representation for Nearest Neighbour Neural Machine Translation // Proc of the 29th International Conference on Computational Linguistics. Stroudsburg, USA: ACL, 2022: 5142-5147.
[27] JIN X Y, GE T, WEI F R. Plug and Play Knowledge Distillation for kNN-LM with External Logits // Proc of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing(Short Papers). Stroudsburg, USA: ACL, 2022: 463-469.
[28] DAI Y H, ZHANG Z R, LIU Q Z, et al. Simple and Scalable Nearest Neighbor Machine Translation[C/OL].[2024-09-10]. https://arxiv.org/pdf/2302.12188.
[29] CAO Z W, YANG B S, LIN H, et al. Bridging the Domain Gaps in Context Representations for k-Nearest Neighbor Neural Machine Translation // Proc of the 61st Annual Meeting of the Association for Computational Linguistics(Long Papers). Stroudsburg, USA: ACL, 2023: 5841-5853.
[30] FAN A, BHOSALE S, SCHWENK H, et al. Beyond English-Ce-ntric Multilingual Machine Translation. Journal of Machine Learning Research, 2021, 22(107): 1-48.
[31] WU L J, TIAN F, QIN T, et al. A Study of Reinforcement Lear-ning for Neural Machine Translation // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2018: 3612-3621.
[32] PAULUS R, XIONG C M, SOCHER R. A Deep Reinforced Model for Abstractive Summarization[C/OL]. [2024-09-10]. https://arxiv.org/pdf/1705.04304.
[33] PAPINENI K, ROUKOS S, WARD T, et al. BLEU: A Method for Automatic Evaluation of Machine Translation // Proc of the 40th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2002: 311-318.
[34] TIEDEMANN J. Parallel Data, Tools and Interfaces in OPUS // Proc of the International Conference on Language Resources and Evaluation. Stroudsburg, USA: ACL, 2012: 2214-2218.
[35] NG N, YEE K, BAEVSKI A, et al. Facebook FAIR's WMT19 News Translation Task Submission // Proc of the 4th Conference on Machine Translation(Shared Task Papers, Day 1). Stroudsburg, USA: ACL, 2019: 314-319.
[36] BAPNA A, ARIVAZHAGAN N, FIRAT O. Simple, Scalable Adaptation for Neural Machine Translation // Proc of the Confe-rence on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg, USA: ACL, 2019: 1538-1548.
[37] 张宝兴,彭敦陆,王雅峰.GEA-NMT:图卷积增强的多任务低资源机器翻译模型.小型微型计算机系统, 2024, 45(9): 2156-2164.
(ZHANG B X, PENG D L, WANG Y F. GEA-NMT: Graph Co-nvolutional Network Enhanced Multi-task Adapting Neural Machine Translation. Journal of Chinese Computer Systems, 2024, 45(9): 2156-2164.)
[38] 朱志国,郭军军,余正涛.一种Mask交互融合预训练知识的低资源神经机器翻译方法.小型微型计算机系统, 2024, 45(3): 591-597.
(ZHU Z G, GUO J J, YU Z T. Low-Resource Neural Machine Translation Method Based on Mask Interactive Fusion of Pretrained Knowledge. Journal of Chinese Computer Systems, 2024, 45(3): 591-597.)
[39] NIU X, MATHUR P, DINU G, et al. Evaluating Robustness to Input Perturbations for Neural Machine Translation // Proc of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2020: 8538-8544.