|
|
|
| Personalized Federated Subgraph Learning with Embedding Alignment and Parameter Activation |
| LU Tianying1,2,3, ZHONG Luying1,2,3, LIAO Shiling1,2,3, YU Zhengxin4, MIAO Wang5, CHEN Zheyi1,2,3 |
1. College of Computer and Data Science, Fuzhou University, Fuzhou 350116;
2. Engineering Research Center of Big Data Intelligence of Ministry of Education, Fuzhou University, Fuzhou 350002;
3. Fujian Key Laboratory of Network Computing and Intelligent Information Processing, Fuzhou University, Fuzhou 350116;
4. School of Computing and Communications, Lancaster University, Lancaster LA1 4YW, UK;
5. Department of Computer Science, University of Exeter, Exeter EX4 4QF, UK |
|
|
|
|
Abstract By integrating subgraph learning and federated learning, federated subgraph learning achieves collaborative learning of subgraph information across multiple clients while protecting data privacy. However, due to different data collection methods of clients, graph data typically exhibits the non-independent and identically distributed(Non-IID) characteristics. Meanwhile, there are significant differences in the structure and features of local graph data across clients. These factors lead to difficult convergence and poor generalization during the training of federated subgraph learning. To solve these problems, a personalized federated subgraph learning framework with embedding alignment and parameter activation(FSL-EAPA) is proposed. First, the personalized model aggregation is performed based on the similarity between clients to reduce the interference of Non-IID data on the overall model performance. Next,the parameter selective activation is introduced during model updates to handle the heterogeneity of subgraph structural features. Finally, the updated client models are utilized to provide positive and negative clustering representations for local node embeddings to aggregate the local nodes with the same class. Thus, FSL-EAPA can fully learn feature representations of nodes, and thereby better adapts to the heterogeneous data distributions across different clients. Experiments on real-world benchmark graph datasets validate the effectiveness of FSL-EAPA. The results show that FSL-EAPA achieves higher classification accuracy under various scenarios.
|
|
Received: 28 February 2025
|
|
|
| Fund:National Natural Science Foundation of China(No.62202103), Central Guidance for Local Science and Technology Development Fund Project(No.2022L3004), Technology and Economy Integration Service Platform of Fujian Province(No.2023XRH001), Fuzhou-Xiamen-Quanzhou National Independent Innovation Demonstration Zone Collaborative Innovation Platform Project(No.2022FX5) |
|
Corresponding Authors:
CHEN Zheyi, Ph.D., professor. His research interests include cloud-edge collaborative computing, resource optimization, deep learning and reinforcement learning.
|
About author:: LU Tianying, Master student. Her research interests include edge computing and federated learning.
ZHONG Luying, Ph.D. candidate. Her research interests include edge computing and federated graph learning.
LIAO Shiling, Master student. Her research interests include edge computing and federated learning.
YU Zhengxin, Ph.D., lecturer. Her research interests include edge computing and federated deep learning.
MIAO Wang, Ph.D., lecturer. His re?search interests include software defined net?works, network function virtualization, and mobile edge computing. |
|
|
|
[1] XIA F, SUN K, YU S, et al. Graph Learning: A Survey. IEEE Transactions on Artificial Intelligence, 2021, 2(2): 109-127.
[2] DEFFERRARD M, BRESSON X, VANDERGHEYNST P. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering // Proc of the 30th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2016: 3844-3852.
[3] ZHANG M H, CHEN Y X.Link Prediction Based on Graph Neural Networks // Proc of the 32nd International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2018: 5171-5181.
[4] ZHANG Z, BU J J, ESTER M, et al. Hierarchical Graph Pooling with Structure Learning[C/OL].[2025-01-16]. https://arxiv.org/pdf/1911.05954.
[5] MCMAHAN B H, MOORE E, RAMAGE D, et al. Communication-Efficient Learning of Deep Networks from Decentralized Data. Proceedings of Machine Learning Research, 2017, 54: 1273-1282.
[6] ZHANG K, YANG C, LI X X, et al.Subgraph Federated Learning with Missing Neighbor Generation // Proc of the 35th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2021: 6671-6682.
[7] WANG B H, LI A, PANG M, et al. GraphFL: A Federated Lear-ning Framework for Semi-supervised Node Classification on Graphs // Proc of the IEEE International Conference on Data Mining. Washington, USA: IEEE, 2022: 498-507.
[8] ZHANG H D, SHEN T, WU F, et al. Federated Graph Learning-A Position Paper[C/OL].[2025-01-16]. https://arxiv.org/pdf/2105.11099.
[9] ZHAO Y, LI M, LAI L Z, et al. Federated Learning with Non-IID Data[C/OL].[2025-01-16]. https://arxiv.org/pdf/1806.00582.
[10] ZHANG X W, HONG M Y, DHOPLE S, et al. FedPD: A Federated Learning Framework with Adaptivity to Non-IID Data. IEEE Transactions on Signal Processing, 2021, 69: 6055-6070.
[11] BRIGGS C, FAN Z, ANDRAS P.Federated Learning with Hierarchical Clustering of Local Updates to Improve Training on Non-IID Data // Proc of the International Joint Conference on Neural Net-works. Washington, USA: IEEE, 2020. DOI: 10.1109/IJCNN4
8605.2020.9207469.
[12] KIPF T N, WELLING M.Semi-supervised Classification with Graph Convolutional Networks[C/OL]. [2025-01-16].https://arxiv.org/pdf/1609.02907.
[13] VELIČKOVIĆ P, CUCURULL G, CASANOVA A, et al. Graph Attention Networks[C/OL].[2025-01-16]. https://arxiv.org/pdf/1710.10903.
[14] HAMILTON W L, YING Z, LESKOVEC J.Inductive Representation Learning on Large Graphs // Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2017: 1025-1035.
[15] XU K, HU W H, LESKOVEC J, et al. How Powerful Are Graph Neural Networks?[C/OL]. [2025-01-16]. https://arxiv.org/pdf/1810.00826.
[16] WU C H, WU F Z, CAO Y, et al. FedGNN: Federated Graph Neural Network for Privacy-Preserving Recommendation[C/OL].[2025-01-16]. https://arxiv.org/pdf/2102.04925.
[17] LIU K, GAO L, KHAN N M, et al. A Multi-stream Graph Convolutional Networks-Hidden Conditional Random Field Model for Skeleton-Based Action Recognition. IEEE Transactions on Multimedia, 2021, 23: 64-76.
[18] BO D Y, WANG X, SHI C, et al. Beyond Low-Frequency Information in Graph Convolutional Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(5): 3950-3957.
[19] LEI M L, QUAN P, MA R R, et al. DigGCN: Learning Compact Graph Convolutional Networks via Diffusion Aggregation. IEEE Transactions on Cybernetics, 2022, 52(2): 912-924.
[20] XU X, WANG T, YANG Y, et al. Radial Graph Convolutional Network for Visual Question Generation. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(4): 1654-1667.
[21] ZHONG L Y, YANG J B, CHEN Z L, et al. Contrastive Graph Convolutional Networks with Generative Adjacency Matrix. IEEE Transactions on Signal Processing, 2023, 71: 772-785.
[22] XIE H, MA J, XIONG L, et al. Federated Graph Classification over Non-IID Graphs // Proc of the 35th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2021: 18839-18852.
[23] HE C Y, CEYANI E, BALASUBRAMANIAN K, et al. Spread-GNN: Serverless Multi-task Federated Learning for Graph Neural Networks[C/OL].[2025-01-16]. https://arxiv.org/pdf/2106.02743.
[24] TAN Y, LIU Y X, LONG G D, et al. Federated Learning on Non-IID Graphs via Structural Knowledge Sharing. Proceedings of the AAAI conference on Artificial Intelligence, 2023, 37(8): 9953-9961.
[25] HUANG W K, WAN G C, YE M, et al. Federated Graph Semantic and Structural Learning // Proc of the 32nd International Joint Conference on Artificial Intelligence. San Francisco, USA: IJCAI, 2023: 3830-3838.
[26] CHEN F W, LONG G D, WU Z H, et al. Personalized Federated Learning with Graph[C/OL].[2025-01-16]. https://arxiv.org/pdf/2203.00829.
[27] PAN Q Y, WU R F, LIU T F, et al. FedGKD: Unleashing the Power of Collaboration in Federated Graph Neural Networks[C/OL].[2025-01-16]. https://arxiv.org/pdf/2309.09517v3.
[28] CHEN F H, LI P, MIYAZAKI T, et al. FedGraph: Federated Graph Learning with Intelligent Sampling. IEEE Transactions on Parallel and Distributed Systems, 2022, 33(8): 1775-1786.
[29] HE C Y, BALASUBRAMANIAN K, CEYANI E, et al. FedGraph-NN: A Federated Learning System and Benchmark for Graph Neural Networks[C/OL].[2025-01-16]. https://arxiv.org/pdf/2104.07145
[30] HOLLAND P W, LASKEY K B, LEINHARDT S.Stochastic Block-models: First Steps. Social Networks, 1983, 5(2): 109-137.
[31] LI T, SAHU A K, ZAHEER M, et al. Federated Optimization in Heterogeneous Networks[C/OL].[2025-01-16]. https://proceedings.mlsys.org/paper_files/paper/2020/file/1f5fe83998a09396ebe6477d9475ba0c-Paper.pdf.
[32] BOJCHEVSKI A, GÜNNEMANN S. Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Ranking[C/OL].[2025-01-16]. https://arxiv.org/pdf/1707.03815.
[33] SHCHUR O, MUMME M, BOJCHEVSKI A, et al. Pitfalls of Graph Neural Network Evaluation[C/OL].[2025-01-16]. https://arxiv.org/pdf/1811.05868.
[34] KARYPIS G, KUMAR V.METIS-Unstructured Graph Partitioning and Sparse Matrix Ordering System, Version 2.0[C/OL].[2025-01-16]. https://www.researchgate.net/publication/246815679_METIS_ _Unstructured_Graph_Partitioning_and_Sparse_Matrix_Ordering_System_Version_20.
[35] BAEK J, JEONG W, JIN J D, et al. Personalized Subgraph Federated Learning. Proceedings of Machine Learning Research, 2023, 202: 1396-1415.
[36] ZHONG L Y, PI Y Y, CHEN Z Y, et al. SpreadFGL: Edge-Client Collaborative Federated Graph Learning with Adaptive Neighbor Generation // Proc of the IEEE Conference on Computer Communications. Washington, USA: IEEE, 2024: 1141-1150.
[37] YAO Y H, JIN W Z, RAVI S, et al.FedGCN: Convergence-Commu-nication Tradeoffs in Federated Training of Graph Convolutional Networks // Proc of the 37th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2023: 79748-79760.
[38] LI X K, WU Z Y, ZHANG W T, et al. FedGTA: Topology-Aware Averaging for Federated Graph Learning. Proceedings of the VLDB Endowment, 2023, 17(1): 41-50.
[39] BLONDEL V D, GUILLAUME J L, LAMBIOTTE R, et al. Fast Unfolding of Communities in Large Networks. Journal of Statistical Mechanics: Theory and Experiment, 2008. DOI: 10.1088/1742-5468/2008/10/P10008. |
|
|
|
2025, Vol. 38 
