Abstract Combining graph convolutional networks(GCN) and auto encoder(AE), a scalable semi-supervised network representation learning model, Semi-GCNAE, is proposed to preserve the network structure information and node feature information. GCN is utilized to capture the structure and feature information of all nodes in K-order neighborhood of the node. The captured information is utilized as the input of AE. The K-order neighborhood information captured by GCN is extracted and the dimension is reduced nonlinearly by AE. The cluster structure information of nodes is preserved by combining Laplacian feature mapping. The ensemble learning method is introduced to train GCN and AE jointly. Therefore, the learned low-dimensional vector representation of nodes can retain both network structure information and node feature information. Extensive evaluation on five real datasets shows that the low-dimensional vector representation of nodes acquired by the proposed model preserves the structure and characteristics of the network effectively. And it generates better performance in node classification, visualization and network reconstruction tasks.
Fund:Supported by Jiangsu Province Production and Research Coope-ration Project(No.BY2015019-30)
About author:: WANG Jie, master student. His research interests include network representation lear-ning.ZHANG Xihuang(Corresponding author), Ph.D., professor. His research interests include computer network, distributed system and application.
WANG Jie,ZHANG Xihuang. Semi-supervised Network Representation Learning Model Based on Graph Convolutional Networks and Auto Encoder[J]. , 2019, 32(4): 317-325.
[1] HOFF P D, RAFTERY A E, HANDCOCK M S. Latent Space Approaches to Social Network Analysis. Technical Report, 399. Seattle, USA: University of Washington, 2001. [2] ZHANG D K, YIN J, ZHU X Q, et al. Network Representation Learning[C/OL]. [2018-11-25]. https://arxiv.org/pdf/1801.05852.pdf. [3] BENGIO Y, COURVILLE A, VINCENT P. Representation Lear-ning: A Review and New Perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 35(8): 1798-1828. [4] BHAGAT S, CORMODE G, MUTHUKRISHNAN S. Node Classification in Social Networks // AGGARWAL C C, ed. Social Network Data Analytics. Berlin, Germany: Springer, 2011: 115-148. [5] VAN DER MAATEN L, HINTON G. Visualizing Data Using t-SNE. Journal of Machine Learning Research, 2008, 9: 2579-2605. [6] TANG J, LIU J Z, ZHANG M, et al. Visualizing Large-Scale and High-Dimensional Data // Proc of the 25th International Conference on World Wide Web. New York, USA: ACM, 2016: 287-297. [7] DING C H Q, HE X F, ZHA H Y, et al. Spectral Min-Max Cut for Graph Partitioning and Data Clustering[C/OL]. [2018-11-25]. https://escholarship.org/uc/item/0g18c027. [8] LIBEN-NOWELL D, KLEINBERG J. The Link-Prediction Problem for Social Networks // Proc of the 20th Annual ACM International Conference on Information and Knowledge Management. New York, USA: ACM, 2003: 556-559. [9] HE K M, ZHANG X Y, REN S Q, et al. Deep Residual Learning for Image Recognition // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 770-778. [10] LEVY O, GOLDBERG Y. Neural Word Embedding as Implicit Matrix Factorization // GHAHARMANI Z, WELLING M, CORTES C, et al., eds. Advances in Neural Information Processing Systems 27. Cambridge, USA: The MIT Press, 2014: 2177-2185. [11] PEROZZI B, AL-RFOU R, SKIENA S. DeepWalk: Online Lear-ning of Social Representations // Proc of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2014: 701-710. [12] GROVER A, LESKOVEC J. Node2vec: Scalable Feature Learning for Networks // Proc of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2016: 855-864. [13] TANG J, QU M, WANG M Z, et al. LINE: Large-Scale Information Network Embedding // Proc of the 24th International Conference on World Wide Web. New York, USA: ACM, 2015: 1067-1077. [14] BRUNA J, ZAREMBA W, SZLAM A, et al. Spectral Networks and Locally Connected Networks on Graphs[C/OL]. [2018-11-25]. https://arxiv.org/pdf/1312.6203.pdf. [15] CHUNG F R K. Spectral Graph Theory. Providence, USA: American Mathematical Society, 1997. [16] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-Based Lear-ning Applied to Document Recognition. Proceedings of the IEEE, 1998, 86(11): 2278-2324. [17] DEFFERRARD M, BRESSON X, VANDERGHEYNST P. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering[C/OL]. [2018-11-25]. https://arxiv.org/pdf/1606.09375.pdf. [18] HAMMOND D K, VANDERGHEYNST P, GRIBONVAL R. Wave-lets on Graphs via Spectral Graph Theory. Applied and Computational Harmonic Analysis, 2011, 30(2): 129-150. [19] KIPF T N, WELLING M. Semi-supervised Classification with Graph Convolutional Networks[C/OL]. [2018-11-25]. https://arxiv.org/pdf/1609.02907.pdf. [20] ZHU X J, GHAHRAMANI Z, LAFFERTY J. Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions // Proc of the 20th International Conference on Machine Learning. Palo Alto, USA: AAAI Press, 2003: 912-919. [21] WANG D X, CUI P, ZHU W. Structural Deep Network Embe-dding // Proc of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2016: 1225-1234. [22] BREIMAN L. Stacked Regression. Machine Learning, 1996, 24(1): 49-64. [23] WOLPERT D H. Stacked Generalization. Neural Networks, 2017, 5(2): 241-259. [24] BELKIN M, NIYOGI P. Laplacian Eigenmaps for Dimensionality Reduction and Data Representation. Neural Computation, 2003, 15(6): 1373-1396. [25] BENGION Y. Learning Deep Architectures for AI. Foundations and Trends in Machine Learning, 2009, 2(1): 1-127. [26] ZHOU D Y, BOUSQUET O, LAL T N, et al. Learning with Local and Global Consistency // Proc of the 16th International Confe-rence on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2003: 321-328 [27] BELKIN M, NIYOGI P, SINDHWANI V. Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples. Journal of Machine Learning Research, 2006, 7: 2399-2434. [28] WESTON J, RATLE F, MOBAHI U, et al. Deep Learning via Semi-supervised Embedding // MONTAVON G, ORR G B, MÜLLER K R, eds. Neural Networks: Tricks of the Trade. Berlin, Germany: Springer, 2012: 639-665. [29] ABADI M, AGARWAL A, BARHAM P, et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems[C/OL]. [2018-11-25]. https://arxiv.org/pdf/1603.04467.pdf. [30] VELICˇKOVIC P, CUCURULL G, CASANOVA A, et al. Graph Attention Networks[C/OL]. [2018-11-25]. https://arxiv.org/pdf/1710.10903.pdf. [31] OU M D, CUI P, PEI J, et al. Asymmetric Transitivity Preserving Graph Embedding // Proc of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2016: 1105-1114 [32] VAN LOAN C F. Generalizing the Singular Value Decomposition. SIAM Journal on Numerical Analysis, 1976, 13(1): 76-83. [33] AHMED A, SHERVASHIDZE N, NARAYANAMURTHY S, et al. Distributed Large-Scale Natural Graph Factorization // Proc of the 22nd International Conference on World Wide Web. New York, USA: ACM, 2013: 37-48.
2019, Vol. 32 
