Abstract The existing Chinese event extraction methods suffer from inadequate modeling of dependencies between an event trigger word and all its corresponding arguments, which results in weakened information interaction within an event and poor performance in argument extraction, especially when there is argument role overlap. To address this issue, a Chinese event extraction method based on graph attention and table pointer network(ATCEE) is proposed in this paper. Firstly,pre-trained character vectors and part-of-speech tagging vectors are fused as feature inputs. Then, the enhanced feature of the event text is obtained by a bidirectional long short-term memory network. Next, a character-level dependency syntax graph is constructed and introduced into multi-layer graph attention network to capture long-range dependencies among constituents of the event text. Subsequently, dependencies between an event trigger word and all its corresponding arguments are further enhanced via a table filling strategy. Finally, the learned table feature is input into a fully connected layer and table pointer network layer for joint extraction of trigger words and arguments. Consequently, long argument entities can be identified better by decoding argument boundaries with a table pointer network. Experimental results indicate that ATCEE method significantly outperforms previous event extraction methods on Chinese benchmark datasets, ACE2005 and DuEE1.0. In addition, the overlap problem of the event argument role is solved by introducing character-level dependency feature and table filling strategy to some extent. The source code of ATCEE can be found at the following website: https://github.com/event6/ATCEE.
Fund:Major Program of National Natural Science Foundation of China(No.61991410), Program of Pujiang National Laboratory(No.P22KN00391)
Corresponding Authors:
LIU Wei, Ph.D., associate professor. His research interests include natural language processing, knowledge representation and reasoning, event ontology and knowledge graph.
About author:: MA Yawei, master student. His research interests include event extraction and natural language processing.PENG Yan, Ph.D., professor. Her research interests include modeling and control of unmanned surface vehicles, field robotics, and locomotion systems.LI Weimin, Ph.D., professor. His research interests include data intelligence, bioinformation, smart medical and social network.
LIU Wei,MA Yawei,PENG Yan等. Chinese Event Extraction Method Based on Graph Attention and Table Pointer Network[J]. Pattern Recognition and Artificial Intelligence, 2023, 36(5): 459-470.
[1] GUAN S P, CHENG X Q, BAI L, et al. What Is Event Knowledge Graph: A Survey. IEEE Transactions on Knowledge and Data Engineering, 2022. DOI: 10.1109/TKDE.2022.3180362. [2] CHEN J H, DU S Y, YANG S. Mining and Evolution Analysis of Network Public Opinion Concerns of Stakeholders in Hot Social Events. Mathematics, 2022, 10(12). DOI: 10.3390/math10122145. [3] LEE M X, MARLOT M. Information Retrieval from Oil and Gas Unstructured Data with Contextualized Framework // Proc of the 3rd EAGE Digitalization Conference and Exhibition. Bunnik, USA: EAGE, 2023. DOI: 10.3997/2214-4609.202332039. [4] WALKER C, STRASSEL S, MEDERO J, et al. ACE 2005 Multilingual Training Corpus. Progress of Theoretical Physics Supplement, 2006, 110(110): 261-276. [5] LI Q, LI J X, SHENG J W, et al. A Survey on Deep Learning Event Extraction: Approaches and Applications. IEEE Transactions on Neural Networks and Learning Systems, 2022. DOI: 10.1109/TNNLS.2022.3213168. [6] CHEN Y B, XU L H, LIU K, et al. Event Extraction via Dynamic Multi-pooling Convolutional Neural Networks // Proc of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Proce-ssing(Long Papers). Stroudsburg, USA: ACL, 2015: 167-176. [7] NGUYEN T H, CHO K, GRISHMAN R. Joint Event Extraction via Recurrent Neural Networks // Proc of the Conference of the North American Chapter of the Association for Computational Linguistics(Human Language Technologies). Stroudsburg, USA: ACL, 2016: 300-309. [8] YANG S, FENG D W, QIAO L B, et al. Exploring Pre-trained Language Models for Event Extraction and Generation // Proc of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2019: 5284-5294. [9] ZENG Y, YANG H H, FENG Y S, et al. A Convolution BiLSTM Neural Network Model for Chinese Event Extraction // Proc of the 5th National CCF Conference on Natural Language Processing and Chinese Computing. Berlin, Germany: Springer, 2016: 275-287. [10] LECUN Y, BOSER B, DENKER J S, et al. Backpropagation Applied to Handwritten Zip Code Recognition. Neural Computation, 1989, 1(4): 541-551. [11] HOCHREITER S, SCHMIDHUBER J. Long Short-Term Memory. Neural Computation, 1997, 9(8): 1735-1780. [12] RAMSHAW L A, MARCUS M P. Text Chunking Using Transformation-Based Learning // ARMSTRONG S, CHURCH K, ISABELLE P, et al., eds. Natural Language Processing Using Very Large Corpora. Berlin, Germany: Springer, 1999: 157-176. [13] DING N, LI Z R, LIU Z Y, et al. Event Detection with Trigger-Aware Lattice Neural Network // Proc of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg, USA: ACL, 2019: 347-356. [14] WU Y, ZHANG J Y. Chinese Event Extraction Based on Attention And Semantic Features: A Bidirectional Circular Neural Network. Future Internet, 2018, 10(10). DOI: 10.3390/fi10100095. [15] XU N, XIE H H, ZHAO D Y. A Novel Joint Framework for Multiple Chinese Events Extraction // Proc of the 19th China National Conference on Chinese Computational Linguistics. Berlin, Germany: Springer, 2020: 174-183. [16] SHENG J W, GUO S, YU B W, et al. CasEE: A Joint Learning Framework with Cascade Decoding for Overlapping Event Extraction // Proc of the Findings of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2021: 164-174. [17] LIU X, LUO Z C, HUANG H Y. Jointly Multiple Events Extraction via Attention-Based Graph Information Aggregation // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2018: 1247-1256. [18] KIPF T N, WELLING M. Semi-Supervised Classification with Gra-ph Convolutional Networks[C/OL]. [2022-2-03].https://arxiv.org/pdf/1609.02907.pdf. [19] YAN H R, JIN X L, MENG X B, et al. Event Detection with Multi-order Graph Convolution and Aggregated Attention // Proc of the Conference on Empirical Methods in Natural Language Proce-ssing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg, USA: ACL, 2019: 5766-5770. [20] YOU H L, SAMUEL D, TOUILEB S, et al. EventGraph: Event Extraction as Semantic Graph Parsing // Proc of the 5th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text. Stroudsburg, USA: ACL, 2022: 7-15. [21] WU X H, WANG T R, FAN Y P, et al. Chinese Event Extraction via Graph Attention Network. ACM Transactions on Asian and Low-Resource Language Information Processing, 2022, 21(4). DOI: 10.1145/3494533. [22] VELIČKOVIĆ P, CUCURULL G, CASANOVA A, et al. Graph Attention Networks[C/OL].[2022-2-03]. https://arxiv.org/pdf/1710.10903.pdf. [23] GUPTA P, SCHÜTZE H, ANDRASSY B. Table Filling Multi-task Recurrent Neural Network for Joint Entity and Relation Extraction // Proc of the 26th International Conference on Computational Linguistics(Technical Papers). Stroudsburg, USA: ACL, 2016: 2537-2547. [24] WU Z, YING C C, ZHAO F, et al. Grid Tagging Scheme for Aspect-Oriented Fine-Grained Opinion Extraction // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2016: 2576-2585. [25] ZENG D J, TIAN J, PENG R Y, et al. Joint Event Extraction Based on Global Event-Type Guidance and Attention Enhancement. Computers, Materials and Continua, 2021, 68(3): 4161-4173. [26] SUN Y, WANG S H, LI Y K, et al. ERNIE: Enhanced Representation through Knowledge Integration[C/OL].[2022-2-03]. https://arxiv.org/pdf/1904.09223.pdf. [27] CHE W X, FENG Y L, Qin L B, et al. N-LTP: An Open-Source Neural Language Technology Platform for Chinese // Proc of the Conference on Empirical Methods in Natural Language Processing(System Demonstrations). Stroudsburg, USA: ACL, 2021: 42-49. [28] LAFFERTY J, MCCALLUM A, PEREIRA F C N. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data // Proc of the 18th International Conference on Machine Learning. San Francisco, USA: Morgan Kaufmann, 2001: 282-289. [29] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding // Proc of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Long and Short Papers). Stroudsburg, USA: ACL, 2019: 4171-4186. [30] VASWANI A, SHAZEER N, PARMAR N, et al. Attention Is All You Need // Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2017: 6000-6010. [31] 许晶航,左万利,梁世宁,等. 基于图注意力网络的因果关系抽取. 计算机研究与发展, 2020, 57(1): 159-174. (XU J H, ZUO W L, LIANG S N, et al. Causal Relation Extraction Based on Graph Attention Networks. Journal of Computer Research and Development, 2020, 57(1): 159-174.) [32] SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the Inception Architecture for Computer Vision // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 2818-2826. [33] MÜLLER R, KORNBLITH S, HINTON G E. When Does Label Smoothing Help? // Proc of the 33rd International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2019: 4694-4703. [34] ZHANG C B, JIANG P T, HOU Q B, et al. Delving Deep into Label Smoothing. IEEE Transactions on Image Processing, 2021, 30: 5984-5996. [35] LOSHCHILOV I, HUTTER F. Decoupled Weight Decay Regularization[C/OL]. [2022-2-03].https://arxiv.org/pdf/1711.05101.pdf. [36] LI X Y, LI F Y, PAN L, et al. DuEE: A Large-Scale Dataset for Chinese Event Extraction in Real-World Scenarios // Proc of the 9th International Conference on Natural Language Processing and Chinese Computing. Berlin, Germany: Springer, 2020: 534-545. [37] GOYAL P, DOLLÁR P, GIRSHICK R, et al. Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour[C/OL].[2022-2-03]. https://arxiv.org/pdf/1706.02677.pdf. [38] KROGH A, HERTZ J. A Simple Weight Decay Can Improve Gene-ralization // Proc of the 4th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 1991: 950-957. [39] 贺瑞芳,段绍杨. 基于多任务学习的中文事件抽取联合模型. 软件学报, 2019, 30(4): 1015-1030. (HE R F, DUAN S Y. Joint Chinese Event Extraction Based Multi-task Learning. Journal of Software, 2019, 30(4): 1015-1030.) [40] KAN Z G, QIAO L B, YANG S, et al. Event Arguments Extraction via Dilate Gated Convolutional Neural Network with Enhanced Local Features. IEEE Access, 2020, 8: 123483-123491. [41] LYU Z H, SHI K J, LI X, et al. Multi-grained Dependency Graph Neural Network for Chinese Open Information Extraction // Proc of the 25th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining. Berlin, Germany: Springer, 2021: 155-167.
2023, Vol. 36 
