Abstract Deep neural networks can be trained to predict age from brain image and the predicted brain age serves as a biomarker for identifying diseases associated with aging. Traditional brain age prediction methods tend to rely on unimodal image data, whereas multimodal data can provide more comprehensive information and improve prediction accuracy. However, existing multimodal fusion methods often fail to fully exploit the correlations and complementarities between different modalities. To overcome these challenges, a collaborative brain age prediction algorithm based on multimodal fuzzy feature fusion(CMFF) is proposed. Fuzzy fusion module and multimodal collaborative convolution module are designed to effectively utilize the correlation and complementary information between the multimodal information. Firstly, feature tensors are extracted from multimodal brain images by a convolutional neural network, and are integrated into a global feature tensor via radial joins. Then, the fuzzy fusion module is employed to learn the fuzzified features, and these features are applied to the multimodal collaborative convolutional module to enhance the complementary information of these features through modality-specific convolutional layers. Finally, the age prediction regression task is performed based on the gender information and the fuzzy collaborative processed features to obtain an accurate predicted age. Experimental results on SRPBS multi-disorder MRI dataset demonstrate the superior performance of CMFF.
Fund:Supported by National Natural Science Foundation of China(No.61976120,62006128,62102199), Natural Science Foundation of Jiangsu Province(No.BK20231337), Double Creation Doctoral Program of Jiangsu Province(No.JSSCBS20230348), Key Program of Natural Science Research of Higher Education of Jiang-su Province(No.21KJA510004),General Program of Natural Science Research of Higher Education of Jiangsu Province(No.23KJB520031), Basic Science Research Project of Nantong Science and Technology Bureau(No.JC2021122), China Postdoctoral Science Foundation(No.2022M711716)
Corresponding Authors:
DING Weiping, Ph.D., professor. His research interests include data mining, machine learning, granular computing, evolutionary computing and big data analytics.
About author:: WANG Jing, Master student. Her research interests include deep learning and fuzzy theory. YIN Tao, Ph.D. candidate. His research interests include granular computing, rough sets and hypergraph neural networks. JU Hengrong, Ph.D., associate professor. His research interests include granular computing, rough sets, machine learning and know-ledge discovery. HUANG Jiashuang, Ph.D., associate pro- fessor. His research interests include brain net- work analysis and deep learning.
WANG Jing,DING Weiping,YIN Tao等. Collaborative Brain Age Prediction Algorithm Based on Multimodal Fuzzy Feature Fusion[J]. Pattern Recognition and Artificial Intelligence, 2024, 37(7): 613-625.
[1] LIU W J, DONG Q X, SUN S T, et al. Risk Prediction of Alzheimer's Disease Conversion in Mild Cognitive Impaired Population Based on Brain Age Estimation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023, 31: 2468-2476. [2] STATSENKO Y, KUZNETSOV N V, MOROZOVA D, et al. Rea-ppraisal of the Concept of Accelerated Aging in Neurodegeneration and Beyond. Cells, 2023, 12(20). DOI: 10.3390/cells122024 51. [3] RAO G X, LI A, LIU Y, et al. A High-Powered Brain Age Prediction Model Based on Convolutional Neural Network // Proc of the IEEE 17th International Symposium on Biomedical Imaging. Wa-shington, USA: IEEE, 2020: 1915-1919. [4] LAM P K, SANTHALINGAM V, SURESH P, et al. Accurate Brain Age Prediction Using Recurrent Slice-Based Networks // Proc of the 16th International Symposium on Medical Information Processing and Analysis. London, UK: SPIE, 2020, 11583: 11-20. [5] FENG X Y, LIPTON Z C, YANG J, et al. Estimating Brain Age Based on a Uniform Healthy Population with Deep Learning and Structural Magnetic Resonance Imaging. Neurobiology of Aging, 2020, 91: 15-25. [6] MOUCHES P, WILMS M, RAJASHEKAR D, et al. Multimodal Bio- logical Brain Age Prediction Using Magnetic Resonance Imaging and Angiography with the Identification of Predictive Regions. Human Brain Mapping, 2022, 43(8): 2554-2566. [7] CAI H J, GAO Y, LIU M H. Graph Transformer Geometric Lear-ning of Brain Networks Using Multimodal MR Images for Brain Age Estimation. IEEE Transactions on Medical Imaging, 2023, 42(2): 456-466. [8] HE S, GRANT P E, OU Y M. Global-Local Transformer for Brain Age Estimation. IEEE Transactions on Medical Imaging, 2022, 41(1): 213-224. [9] LUND M J, ALNæS D, DE LANGE A M G, et al. Brain Age Prediction Using fMRI Network Coupling in Youths and Associations with Psychiatric Symptoms. NeuroImage: Clinical, 2022, 33. DOI: 10.1016/j.nicl.2021.102921. [10] LI H M, SATTERTHWAITE T D, FAN Y. Brain Age Prediction Based on Resting-State Functional Connectivity Patterns Using Convolutional Neural Networks // Proc of the IEEE 15th International Symposium on Biomedical Imaging. Washington, USA: IEEE, 2018: 101-104. [11] ZHAI J, LI K. Predicting Brain Age Based on Spatial and Temporal Features of Human Brain Functional Networks. Frontiers in Human Neuroscience, 2019, 13. DOI: 10.3389/fnhum.2019.00062. [12] GUAN S H, JIANG R Z, MENG C, et al. Brain Age Prediction across the Human Lifespan Using Multimodal MRI Data. GeroScience, 2024, 46(1): 1-20. [13] HUANG J Y, KE P F, CHEN X Y, et al. Multimodal Magnetic Resonance Imaging Reveals Aberrant Brain Age Trajectory during Youth in Schizophrenia Patients. Frontiers in Aging Neuroscience, 2022, 14. DOI: 10.3389/fnagi.2022.823502. [14] 林岚,熊敏.英国生物银行在神经影像领域应用的研究综述.生物医学工程学杂志, 2021, 38(3): 594-601. (LIN L, XIONG M. A Review on the Application of UK Biobank in Neuroimaging. Journal of Biomedical Engineering, 2021, 38(3): 594-601.) [15] TANVEER M, GANAIE M A, BEHESHTI I, et al. Deep Learning for Brain Age Estimation: A Systematic Review. Information Fusion, 2023, 96: 130-143. [16] COLE J H, POUDEL R P K, TSAGKRASOULIS D, et al. Predicting Brain Age with Deep Learning from Raw Imaging Data Results in a Reliable and Heritable Biomarker. NeuroImage, 2017, 163: 115-124. [17] PENG H, GONG W K, BECKMANN C F, et al. Accurate Brain Age Prediction with Lightweight Deep Neural Networks. Medical Image Analysis, 2021, 68. DOI: 10.1016/j.media.2020.1018 71. [18] NING K D, DUFFY B A, FRANKLIN M, et al. Improving Brain Age Estimates with Deep Learning Leads to Identification of Novel Genetic Factors Associated with Brain Aging. Neurobiology of Aging, 2021, 105: 199-204. [19] HU Y X, WANG H L, LI B B. SQET: Squeeze and Excitation Transformer for High-Accuracy Brain Age Estimation // Proc of the IEEE International Conference on Bioinformatics and Biomedicine. Washington, USA: IEEE, 2022: 1554-1557. [20] POLONI K M, FERRARI R J. A Deep Ensemble Hippocampal CNN Model for Brain Age Estimation Applied to Alzheimer's Diagnosis. Expert Systems with Applications, 2022, 195. DOI: 10.1016/j.eswa.2022.116622. [21] LEE J, BURKETT B J, MIN H K, et al. Deep Learning-Based Brain Age Prediction in Normal Aging and Dementia. Nature Aging, 2022, 2(5): 412-424. [22] MONTALÀ-FLAQUER M, CAÑETE-MASSÉ C, VAQUÉ-ALCÁ-ZAR L, et al. Spontaneous Brain Activity in Healthy Aging: An Overview Through Fluctuations and Regional Homogeneity. Frontiers in Aging Neuroscience, 2023, 14. DOI: 10.3389/fnagi.2022.1002811. [23] DENG S W, FRANKLIN C G, O'BOYLE M, et al. Hemodynamic and Metabolic Correspondence of Resting-State Voxel-Based Physiological Metrics in Healthy Adults. NeuroImage, 2022, 250. DOI: 10.1016/j.neuroImage.2022.118923. [24] MOUCHES P, WILMS M, AULAKH A, et al. Multimodal Brain Age Prediction Fusing Morphometric and Imaging Data and Asso-ciation with Cardiovascular Risk Factors. Frontiers in Neurology, 2022, 14. DOI: 10.3389/fneur.2022.979774. [25] CAI H S, QU Z D, LI Z, et al. Feature-Level Fusion Approaches Based on Multimodal EEG Data for Depression Recognition. Information Fusion, 2020, 59: 127-138. [26] LIU Y, MU F H, SHI Y, et al. Brain Tumor Segmentation in Multimodal MRI via Pixel-Level and Feature-Level Image Fusion. Frontiers in Neuroscience, 2022, 16. DOI: 10.3389/fnins.2022.1000587. [27] ZHAO H Y, CAI H J, LIU M H. Transformer Based Multi-modal MRI Fusion for Prediction of Post-Menstrual Age and Neonatal Brain Development Analysis. Medical Image Analysis, 2024, 94. DOI: 10.1016/j.media.2024.103140. [28] RALLABANDI V P S, SEETHARAMAN K. Deep Learning-Based Classification of Healthy Aging Controls, Mild Cognitive Impairment and Alzheimer's Disease Using Fusion of MRI-PET Imaging. Biomedical Signal Processing and Control, 2023, 80. DOI: 10.1016/j.bspc.2022.104312. [29] GUO S C, WANG L H, CHEN Q J, et al. Multimodal MRI Image Decision Fusion-Based Network for Glioma Classification. Frontiers in Oncology, 2022, 12. DOI: 10.3389/fonc.2022.819673. [30] ASGHARZADEH-BONAB A, KALBKHANI H, AZARFARDIAN S. An Alzheimer's Disease Classification Method Using Fusion of Features from Brain Magnetic Resonance Image Transforms and Deep Convolutional Networks. Healthcare Analytics, 2023, 4. DOI: 10.1016/j.health.2023.100223. [31] SHOEIBI A, KHODATARS M, JAFARI M, et al. Diagnosis of Brain Diseases in Fusion of Neuroimaging Modalities Using Deep Learning: A Review. Information Fusion, 2023, 93: 85-117. [32] SONG J, ZHENG J, LI P, et al. An Effective Multimodal Image Fusion Method Using MRI and PET for Alzheimer's Disease Diagnosis. Frontiers in Digital Health, 2021, 3. DOI: 10.3389/fdgth.2021.637386. [33] ZHOU T, FU H Z, CHEN G, et al. Hi-Net: Hybrid-Fusion Network for Multi-modal MR Image Synthesis. IEEE Transactions on Medical Imaging, 2020, 39(9): 2772-2781. [34] DE DIAS M F M, CARVALHO P, DUARTE J V, et al. Deformation Fields: A New Source of Information to Predict Brain Age. Journal of Neural Engineering, 2022, 19(3). DOI: 10.1088/174 1-2552/ac7003. [35] GANDHI A, ADHVARYU K, PORIA S, et al. Multimodal Sentiment Analysis: A Systematic Review of History, Datasets, Multimodal Fusion Methods, Applications, Challenges and Future Directions. Information Fusion, 2023, 91: 424-444. [36] NIE W Z, YAN Y, SONG D, et al. Multi-modal Feature Fusion Based on Multi-layers LSTM for Video Emotion Recognition. Multimedia Tools and Applications, 2021, 80: 16205-16214. [37] DEY S, BHATTACHARYA R, MALAKAR S, et al. Choquet Fu- zzy Integral-Based Classifier Ensemble Technique for COVID-19 Detection. Computers in Biology and Medicine, 2021, 135. DOI: 10.1016/j.compbiomed.2021.104585. [38] BHOWAL P, SEN S, YOON J H, et al. Choquet Integral and Coalition Game-Based Ensemble of Deep Learning Models for COVID-19 Screening from Chest X-Ray Images. IEEE Journal of Biomedical and Health Informatics, 2021, 25(12): 4328-4339. [39] XUE Y H, CHEN R, WANG J G, et al. Choquet Integral-Based Multimodal Fusion Strategy in the Application of Atherosclerosis Risk Prediction // Proc of the IEEE 12th Data Driven Control and Learning Systems Conference. Washington, USA: IEEE, 2023: 1847-1852. [40] PALANISAMI D, MOHAN N, GANESHKUMAR L, et al. A New Approach of Multi-modal Medical Image Fusion Using Intuitionistic Fuzzy Set. Biomedical Signal Processing and Control, 2022, 77. DOI: 10.1016/j.bspc.2022.103762. [41] TIRUPAL T, MOHAN B C, KUMAR S S. Multimodal Medical Image Fusion Techniques-A Review. Current Signal Transduction Therapy, 2021, 16(2): 142-163. [42] DENG J X, DENG Y. Information Volume of Fuzzy Membership Function. International Journal of Computers Communications and Control, 2021, 16(1). DOI: 10.15837/ijccc.2021.1.4106. [43] ZHANG D L, MESIAR R, PAP E. Pseudo-Integral and Genera-lized Choquet Integral. Fuzzy Sets and Systems, 2022, 446: 193-221. [44] ISLAM M A, ANDERSON D T, PINAR A J, et al. Enabling Explainable Fusion in Deep Learning with Fuzzy Integral Neural Networks. IEEE Transactions on Fuzzy Systems, 2020, 28(7): 1291-1300. [45] CHENG J, LIU Z Y, GUAN H, et al. Brain Age Estimation from MRI Using Cascade Networks with Ranking Loss. IEEE Transactions on Medical Imaging, 2021, 40(12): 3400-3412. [46] DÖRFEL R P, ARENAS-GOMEZ J M, FISHER P M, et al. Prediction of Brain Age Using Structural Magnetic Resonance Imaging: A Comparison of Accuracy and Test-Retest Reliability of Publicly Available Software Packages. Human Brain Mapping, 2023, 44(17): 6139-6148. [47] GONG W K, BECKMANN C F, VEDALDI A, et al. Optimising a Simple Fully Convolutional Network for Accurate Brain Age Prediction in the PAC 2019 Challenge. Frontiers in Psychiatry, 2021, 12. DOI: 10.3389/fpsyt.2021.627996. [48] TANAKA S C, YAMASHITA A, YAHATA N, et al. A Multi-site, Multi-disorder Resting-State Magnetic Resonance Image Database. Scientific Data, 2021, 8(1): 1-15.
2024, Vol. 37 
