基于门控双层异构图注意力网络的半监督短文本分类

doi:10.16451/j.cnki.issn1003-6059.202307003

Abstract
Figure/Table
References (0)
Related Citation (15)

Download: PDF (892 KB) HTML (1 KB)
Export: BibTeX | EndNote (RIS)

Abstract

To address the issues of insufficient utilization of information between nodes and overfitting in short text classification based on heterogeneous graph neural network, a method for semi-supervised short text classification based on gated double-layer heterogeneous graph attention network(GDHG) is proposed. GDHG consists of two layers: node attention and gated heterogeneous graph attention network. Firstly, different types of node attention coefficients are trained by node attention, and then the node attention coefficient is input into the gated heterogeneous graph attention network to obtain the gated double-layer attention. Secondly, the gated double-layer attention is multiplied by different states of the nodes to acquire the aggregated node features. Finally, the short texts are classified with the softmax function. In the proposed GDHG, the information forgetting mechanism of node attention and gated heterogeneous graph attention network is utilized to aggregate node information. Consequently, the information of neighboring nodes is effectively obtained. And then the hidden information of different neighboring nodes is mined to improve the ability to aggregate information from remote nodes. Experiment on four short text datasets , Twitter, MR, Snippets and AGNews, illustrate the superiority of GDHG.

Key words： Gated Heterogeneous Graph Attention Semi-Supervised Learning Heterogeneous Graph Neural Network Short Text Classification

Received: 06 December 2022

ZTFLH:	TP181
	TP391

Fund:

Regional Joint Fund for Innovation and Development of National Natural Science Foundation of China(No.U22A20102), “Pioneer” and “Leading Goose” Research and Deve-lopment Program of Zhejiang Province (No.2023C01150)

Corresponding Authors: ZHANG Xiongtao, Ph.D., associate professor. His research interests include artificial intelligence, pattern recognition and machine learning.

About author:: JIANG Yunliang, Ph.D., professor. His research interests include intelligent information processing and geographic information system. WANG Qingpeng, master. His research interests include natural language processing and machine learning. HUANG Xu, Ph.D., associate professor. His research interests include artificial inte-lligence, pattern recognition and machine learning. SHEN Qing, master, professor. Her research interests include intelligent information processing. RAO Jiafeng, master. His research inte-rests include intelligent information processing and machine learning.

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	JIANG Yunliang
	WANG Qingpeng
	ZHANG Xiongtao
	HUANG Xu
	SHEN Qing
	RAO Jiafeng

Cite this article:

JIANG Yunliang,WANG Qingpeng,ZHANG Xiongtao等. Semi-Supervised Short Text Classification Based on Gated Double-Layer Heterogeneous Graph Attention Network[J]. Pattern Recognition and Artificial Intelligence, 2023, 36(7): 602-612.

URL:

http://manu46.magtech.com.cn/Jweb_prai/EN/10.16451/j.cnki.issn1003-6059.202307003 OR http://manu46.magtech.com.cn/Jweb_prai/EN/Y2023/V36/I7/602

[1] AGGARWAL C C, ZHAI C X. A Survey of Text Classification Algorithms // AGGARWAL C C, ZHAI C X, eds. Mining Text Data. Berlin, Germany: Springer, 2012: 163-222.
[2] WANG J, WANG Z Y, ZHANG D W, et al. Combining Knowledge with Deep Convolutional Neural Networks for Short Text Classification // Proc of the 26th International Joint Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2017: 2915-2921.
[3] WANG X, CHEN R H, JIA Y, et al. Short Text Classification Using Wikipedia Concept Based Document Representation // Proc of the International Conference on Information Technology and Applications. Washington, USA: IEEE, 2013: 471-474.
[4] LI X H, YAN L, QIN N, et al. A Novel Semi-Supervised Short Text Classification Algorithm Based on Fusion Similarity // Proc of the International Conference on Intelligent Computing. Berlin, Germany: Springer, 2017: 309-319.
[5] LEE J H, KO S, HAN Y S. SALNet: Semi-Supervised Few-Shot Text Classification with Attention-Based Lexicon Construction. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(14): 13189-13197.
[6] JOHNSON R, ZHANG T. Semi-Supervised Convolutional Neural Net-works for Text Categorization via Region Embedding // Proc of the 28th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2015: 919-927.
[7] CHEN L, ZHANG M C, FU Z B, et al. FLiText: A Faster and Lighter Semi-Supervised Text Classification with Convolution Networks // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2021: 2481-2491.
[8] WANG Y Q, WANG S, YAO Q M, et al. Hierarchical Heteroge-neous Graph Representation Learning for Short Text Classification // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2021: 3091-3101.
[9] LIU Y H, GUAN R C, GIUNCHIGLIA F, et al. Deep Attention Diffusion Graph Neural Networks for Text Classification // Proc of the Conference on Empirical Methods in Natural Language Proce-ssing. Stroudsburg, USA: ACL, 2021: 8142-8152.
[10] YAO L, MAO C S, LUO Y. Graph Convolutional Networks for Text Classification // Proc of the 33rd AAAI Conference on Artificial Intelligence and 31st Innovative Applications of Artificial Inte-lligence Conference and 9th AAAI Symposium on Educational Advances in Artificial Intelligence. Palo Alto, USA: AAAI Press, 2019: 7370-7377.
[11] KIPF T N, WELLING M. Semi-Supervised Classification with Graph Convolutional Networks[C/OL]. [2022-12-15]. https://arxiv.org/pdf/1609.02907.pdf.
[12] LIU X E, YOU X X, ZHANG X, et al. Tensor Graph Convolutional Networks for Text Classification. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(5): 8409-8416.
[13] 宋泽宇,李旸,李德玉,等.融合标签关系的法律文本多标签分类方法.模式识别与人工智能. 2022, 35(2): 185-192.
(SONG Z Y, LI Y, LI D Y, et al. Multi-label Classification of Legal Text with Fusion of Label Relations. Pattern Recognition and Artificial Intelligence, 2022, 35(2): 185-192.)
[14] DING K Z, WANG J L, LI J D, et al. Be More with Less: Hypergraph Attention Networks for Inductive Text Classification // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2020: 4927-4936.
[15] YANG T C, HU L M, SHI C, et al. HGAT: Heterogeneous Graph Attention Networks for Semi-Supervised Short Text Classification. ACM Transactions on Information Systems, 2021, 39(3). DOI: 10.1145/3450352.
[16] LI Y J, ZEMEL R, BROCKSCHMIDT M, et al. Gated Graph Sequence Neural Networks[C/OL].[2022-12-15]. https://arxiv.org/pdf/1511.05493.pdf.
[17] SUN Y Z, HAN J W. Mining Heterogeneous Information Networks: A Structural Analysis Approach. ACM SIGKDD Explorations Newsletter, 2013, 14(2): 20-28.
[18] BLEI D M, NG A Y, JORDAN M I. Latent Dirichlet Allocation. Journal of Machine Learning Research, 2003, 3: 993-1022.
[19] PANG B, LEE L. Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales // Proc of the 43rd Annual Meeting of Association for Computational Linguistics. Stroudsburg, USA: ACL, 2005: 115-124.
[20] PHAN X H, NGUYEN L M, HORIGUCHI S. Learning to Classify Short and Sparse Text & Web with Hidden Topics from Large-Scale Data Collections // Proc of the 17th International Conference on World Wide Web. New York, USA: ACM, 2008: 91-100.
[21] ZHANG X, ZHAO J B, LECUN Y. Character-Level Convolutional Networks for Text Classification // Proc of the 28th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2015: 649-657.
[22] DRUCKER H, WU D H, VAPNIK V N. Support Vector Machines for Spam Categorization. IEEE Transactions on Neural Networks, 1999, 10(5): 1048-1054.
[23] COVER T, HART P. Nearest Neighbor Pattern Classification. IEEE Transactions on Information Theory, 1967, 13(1): 21-27.
[24] LIU P F, QIU X P, HUANG X J. Recurrent Neural Network for Text Classification with Multi-task Learning // Proc of the 25th International Joint Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2016: 2873-2879.
[25] JOULIN A, GRAVE E, BOJANOWSKI P, et al. Bag of Tricks for Efficient Text Classification // Proc of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2017: 427-431.
[26] KIM Y. Convolutional Neural Networks for Sentence Classification // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2014: 1746-1751.
[27] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding // Proc of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies(Long and Shot Paper). Stroudsburg, USA: ACL, 2019: 4171-4186.
[28] CHEN J D, HU Y Z, LIU J P, et al. Deep Short Text Classification with Knowledge Powered Attention // Proc of the 33rd AAAI Conference on Artificial Intelligence and 31st Innovative Applications of Artificial Intelligence Conference and 9th AAAI Symposium on Educational Advances in Artificial Intelligence. Palo Alto, USA: AAAI Press, 2019: 6252-6259.
[29] VELI?KOVI? P, CUCURULL G, CASANOVA A, et al. Graph Attention Networks[C/OL].[2022-12-15]. https://arxiv.org/pdf/1710.10903.pdf.