Abstract：In most learning models for knowledge graph representation, only structural knowledge between entities and relations is taken into account. Therefore, the capability of the models is limited by knowledge storage, and the completion performance of knowledge base is unstable. Existing knowledge representation methods incorporating external information mostly model for a specific kind of external modal information, leading to limited application scopes. Thus, a knowledge representation learning model, Conv-AT, is proposed. Firstly, two external modes of information, text and images, are considered, and three schemes fusing external knowledge and entities are introduced to obtain multimodal representation of entities. Secondly, the performance of convolution is enhanced and the quality of knowledge representation as well as the completion ability of the model are improved by combining the channel attention module and spatial attention module. Link prediction and triple classification experiments are conducted on public multimodal datasets, and the results show that the proposed method is superior to other methods.
汪璟玢, 苏华, 赖晓连. 基于多模态表示学习的知识库补全[J]. 模式识别与人工智能, 2021, 34(1): 33-43.
WANG Jingbin, SU Hua, LAI Xiaolian. Knowledge Base Completion Based on Multimodal Representation Learning. , 2021, 34(1): 33-43.
[1] GUAN N N, SONG D D, LIAO L J. Knowledge Graph Embedding with Concepts. Knowledge-Based Systems, 2019, 164: 38-44. [2] MILLER G A. WordNet: A Lexical Database for English. Communications of the ACM, 1995, 38(11): 39-41. [3] BOLLACKER K, EVANS C, PARITOSH P, et al. Freebase: A Collaboratively Created Graph Database for Structuring Human Knowledge // Proc of the ACM SIGMOD International Conference on Management of Data. New York, USA: ACM, 2008: 1247-1250. [4] AUER S, BIZER C, KOBILAROV G, et al. DBpedia: A Nucleus for a Web of Open Data // Proc of the Asian Semantic Web Conference. Berlin, Germany: Springer, 2007: 722-735. [5] SUCHANEK F M, KASNECI G, WEIKUM G. Yago: A Core of Semantic Knowledge // Proc of the 16th International Conference on World Wide Web. New York, USA: ACM, 2007: 697-706. [6] BORDES A, WESTON J, COLLOBERT R, et al. Learning Structured Embeddings of Knowledge Bases // Proc of the 25th AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2011: 301-306. [7] SOCHER R, CHEN D Q, MANNING C D, et al. Reasoning with Neural Tensor Networks for Knowledge Base Completion // Proc of the 26th International Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2013: 926-934. [8] JI S X, PAN S R, CAMBRIA E, et al. A Survey on Knowledge Graphs: Representation, Acquisition and Applications[C/OL]. [2020-09-24]. https://arxiv.org/pdf/2002.00388v2.pdf. [9] BORDES A, USUNIER N, GARCIA-DURAN A, et al. Translating Embeddings for Modeling Multi-relational Data // Proc of the 26th International Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2013: 2787-2795. [10] NICKEL M, TRESP V, KRIEGEL H P. A Three-Way Model for Collective Learning on Multi-relational Data // Proc of the 28th International Conference on Machine Learning. Madison, USA: Omnipress, 2011: 809-816. [11] DETTMERS T, MINERVINI P, STENETORP P, et al. Convolutional 2D Knowledge Graph Embeddings // Proc of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2018: 1811-1818. [12] XIE R B, LIU Z Y, JIA J, et al. Representation Learning of Knowledge Graphs with Entity Descriptions // Proc of the 30th AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2016: 2659-2665. [13] XIAO H, HUANG M, MENG L, et al. SSP: Semantic Space Projection for Knowledge Graph Embedding with Text Descriptions // Proc of the 31st AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2017: 3104-3110. [14] VEIRA N, KENG B, PADMANABHAN K, et al. Unsupervised Embedding Enhancements of Knowledge Graphs Using Textual Associations // Proc of the 28th International Joint Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2019: 5218-5225. [15] XIE R B, LIU Z Y, LUAN H B, et al. Image-Embodied Know-ledge Representation Learning // Proc of the 26th International Joint Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2016: 3140-3146. [16] MOUSSELLY-SERGIEH H, BOTSCHEN T, GUREVYCH I, et al. A Multimodal Translation-Based Approach for Knowledge Graph Representation Learning // Proc of the 7th Joint Conference on Lexical and Computational Semantics. Stroudsburg, USA: ACL, 2018: 225-234. [17] ROTHE S, SCHÜTZE H. AutoExtend: Extending Word Embe-ddings to Embeddings for Synsets and Lexemes // Proc of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing(Long Papers). Stroudsburg, USA: ACL, 2015: 1793-1803. [18] SIMONYAN K, ZISSERMAN A. Very Deep Convolutional Networks for Large-Scale Image Recognition[C/OL]. [2020-09-24]. https://arxiv.org/pdf/1409.1556.pdf. [19] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient Estimation of Word Representations in Vector Space[C/OL]. [2020-09-24]. https://arxiv.org/pdf/1301.3781.pdf. [20] CHATFIELD K, SIMONYAN K, VEDALDI A, et al. Return of the Devil in the Details: Delving Deep into Convolutional Nets[C/OL]. [2020-09-24]. https://arxiv.org/pdf/1405.3531.pdf.
2021, Vol. 34 
