Chain Entity Relation Extraction Model with Filtering Mechanism
XIA Hongbin1,2, SHEN Jian1, LIU Yuan1,2
1. School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122; 2. Jiangsu Key Laboratory of Media Design and Software Technology, Jiangnan University, Wuxi 214122
Abstract Stacking labeling layer is commonly adopted to deal with relation overlap in current entity relation extraction task. In this method, the calculation of the labeling layers corresponding to many relations is redundant, resulting in sparse labeling matrix and weak extraction performance of the model. To solve these problems, a chain entity relation extraction model with filtering mechanism is proposed. Firstly, the vector feature of the text is obtained through the encoding layer, then the subject, object and relation of the relation triple are sequentially extracted through the five-stage chain decoding structure. The chain decoding structure avoids the sparse labeling matrix, and the automatic alignment of entities and relations is completed through the filtering mechanism. In the decoding process, conditional layer normalization is employed to improve the fusion degree of features between stages and reduce the impact of error accumulation. Gated unit is utilized to optimize the fitting performance of the model. Head-to-tail separation and relation correction module are applied to multiple verification of relation sets. Comparative experiments on public datasets show that the proposed model achieves better performance.
Fund:National Natural Science Foundation of China(No.61972182)
Corresponding Authors:
XIA Hongbin, Ph.D., associate professor. His research interests include personalized recommendation systems and natural language processing.
About author:: SHEN Jian, master student. His research interests include natural language proce-ssing. LIU Yuan, master, professor. His research interests include cyber security and social network.
[1] 张勇,高大林,巩敦卫,等.用于关系抽取的注意力图长短时记忆神经网络.智能系统学报, 2021, 16(3): 518-527. (ZHANG Y, GAO D L, GONG D W, et al. Attention Graph Long Short-Term Memory Neural Network for Relation Extraction. CAAI Transactions on Intelligent Systems, 2021, 16(3): 518-527.) [2] HAN X, LIU Z Y, SUN M S. Neural Knowledge Acquisition via Mutual Attention between Knowledge Graph and Text. Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018, 32(1): 4832-4839. [3] LUAN Y, HE L H, OSTENDORF M, et al. Multi-task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2018: 3219-3232. [4] 孙明俊,张丹,郑明智,等.基于人工智能的类风湿性关节炎中医辅助诊疗系统.模式识别与人工智能, 2021, 34(4): 343-352. (SUN M J, ZHANG D, ZHENG M Z, et al. Traditional Chinese Medicine Aided Diagnosis and Treatment System for Rheumatoid Arthritis Based on Artificial Intelligence. Pattern Recognition and Artificial Intelligence, 2021, 34(4): 343-352.) [5] 夏鸿斌,李强,肖奕飞.用于方面情感三元组抽取的词对关系学习方法.模式识别与人工智能, 2022, 35(3): 262-270. (XIA H B, LI Q, XIAO Y F. Word-Pair Relation Learning Method for Aspect Sentiment Triplet Extraction. Pattern Recognition and Artificial Intelligence, 2022, 35(3): 262-270.) [6] ZHENG S C, WANG F, BAO H Y, et al. Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme // Proc of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2017: 1227-1236. [7] DIXIT K, Al-ONAIZAN Y. Span-Level Model for Relation Extraction // Proc of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2019: 5308-5314. [8] EBERTS M, ULGES A. Span-Based Joint Entity and Relation Extraction with Transformer Pre-training[C/OL]. [2023-04-16]. https://arxiv.org/pdf/1909.07755.pdf. [9] WEI Z P, SU J L, WANG Y, et al. A Novel Cascade Binary Ta-gging Framework for Relational Triple Extraction // Proc of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2020: 1476-1488. [10] 王泽儒,柳先辉.基于指针级联标注的中文实体关系联合抽取模型.武汉大学学报(理学版), 2022, 68(3): 304-310. (WANG Z R, LIU X H. Joint Model of Chinese Entity-Relation Extraction Based on a Pointer Cascade Tagging Strategy. Journal of Wuhan University(Natural Science Edition), 2022, 68(3): 304-310.) [11] KAMBHATLA N. Combining Lexical, Syntactic, and Semantic Features with Maximum Entropy Models for Information Extraction // Proc of the ACL Interactive Poster and Demonstration Sessions. Stroudsburg, USA: ACL, 2004: 178-181. [12] HUA W Z, DAI Z H, LIU H X, et al. Transformer Quality in Li-near Time // Proc of the 39th International Conference on Machine Learning. San Diego, USA: JMLR, 2022: 9099-9117. [13] WANG Y C, YU B W, ZHANG Y Y, et al. TPLinker: Single-Stage Joint Extraction of Entities and Relations through Token Pair Linking // Proc of the 28th International Conference on Computational Linguistics. Stroudsburg, USA: ACL, 2020: 1572-1582. [14] ZHENG H Y, WEN R, CHEN X, et al. PRGC: Potential Relation and Global Correspondence Based Joint Relational Triple Extraction // Proc of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confe-rence on Natural Language Processing. Stroudsburg, USA: ACL, 2021: 6225-6235. [15] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding // Proc of the Conference of the North American Chapter of the Association for Computational Linguistics(Human Language Technologies). Stroudsburg, USA: ACL, 2019: 4171-4186. [16] 张龙辉,尹淑娟,任飞亮,等.BSLRel:基于二元序列标注的级联关系三元组抽取模型.中文信息学报, 2021, 35(6): 74-84. (ZHANG L H, YIN S J, REN F L, et al. BSLRel: A Binary Sequence Labeling Based Cascading Relation Triple Extraction Mo-del. Journal of Chinese Information Processing, 2021, 35(6): 74-84.) [17] DAUPHIN Y N, FAN A, AULI M, et al. Language Modeling with Gated Convolutional Networks // Proc of the 34th International Conference on Machine Learning. New York, USA: ACM, 2017: 933-941. [18] VASWANI A, SHAZEER N, PARMAR N, et al. Attention Is All you Need // Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2017: 6000-6010. [19] SU J L, LU Y, PAN S F, et al. Roformer: Enhanced Transformer with Rotary Position Embedding[C/OL].[2023-04-16]. https://arxiv.org/pdf/2104.09864.pdf. [20] RIEDEL S, YAO L M, MCCALLUM A. Modeling Relations and Their Mentions without Labeled Text // Proc of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Berlin, Germany: Springer, 2010: 148-163. [21] GARDENT C, SHIMORINA A, NARAYAN S, et al. Creating Training Corpora for NLG Micro-Planners// Proc of the 55th Annual Meeting of the Association for Computational Linguistics(Long Papers). Stroudsburg, USA: ACL, 2017: 179-188. [22] FU T J, LI P H, MA W Y. GraphRel: Modeling Text as Relational Graphs for Joint Entity and Relation Extraction // Proc of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2019: 1409-1418. [23] ZENG X R, HE S Z, ZENG D J, et al. Learning the Extraction Order of Multiple Relational Facts in a Sentence with Reinforcement Learning // Proc of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Confe-rence on Natural Language Processing. Stroudsburg, USA: ACL, 2019: 367-377. [24] YU B W, ZHANG Z Y, SHU X B, et al. Joint Extraction of Entities and Relations Based on a Novel Decomposition Strategy[C/OL].[2023-04-16]. https://arxiv.org/pdf/1909.04273.pdf. [25] SUI D B, CHEN Y B, LIU K, et al. Joint Entity and Relation Extraction with Set Prediction Networks[C/OL].[2023-04-16]. https://arxiv.org/pdf/2011.01675.pdf. [26] XU B F, WANG Q, LYU Y, et al. EmRel: Joint Representation of Entities and Embedded Relations for Multi-triple Extraction // Proc of the Conference of the North American Chapter of the Association for Computational Linguistics(Human Language Technologies). Stroudsburg, USA: ACL, 2022: 659-665. [27] SHANG Y M, HUANG H Y, MAO X L. OneRel: Joint Entity and Relation Extraction with One Module in One Step. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(10): 11285-11293.
2023, Vol. 36 
