Abstract Convolutional neural networks make certain progress in medical image segmentation tasks due to their powerful feature extraction capabilities. However, the accuracy of edge segmentation still needs to be improved. To address this problem, a triplet-path network based on edge selection graph reasoning is proposed in this paper, including the target localization path, edge selection path and refinement path. In the target localization path, a multi-scale feature fusion module is designed to aggregate high-level features for the localization of lesion regions. In the edge selection path, an edge-selective graph reasoning module is constructed for edge screening of low-level features and graph reasoning to ensure the edge shape of the relevant lesion region. In the refinement path, a progressive group level refinement module is established to refine the structure information and details of different scale features. Moreover, a composite loss fusing weighted Focal Tversky loss and a weighted intersection over union loss is introduced to mitigate the effects of class imbalance. Experimental results on public datasets demonstrate the superior performance of the proposed method.
Fund:National Natural Science Foundation of China(No.62006215,62176244)
Corresponding Authors:
YE Hailiang, Ph.D., lecturer. His research interests include deep learning, graph neural networks, image processing and point cloud analysis.
About author:: CAO Feilong, Ph.D., professor. His research interests include deep learning and image processing.JIANG Qingting, Master student. Her research interests include deep learning, graph neural networks and medical image proce-ssing.
[1] LITJENS G, KOOI T, BEJNORDI B E, et al. A Survey on Deep Learning in Medical Image Analysis. Medical Image Analysis, 2017, 42: 60-88. [2] 刘少鹏,洪佳明,梁杰鹏,等.面向医学图像分割的半监督条件生成对抗网络.软件学报, 2020, 31(8): 2588-2602. (LIU S P, HONG J M, LIANG J P, et al. Medical Image Segmentation Using Semi-Supervised Conditional Generative Adversarial Nets. Journal of Software, 2020, 31(8): 2588-2602.) [3] SHEN D G, WU G R, SUK H I.Deep Learning in Medical Image Analysis. Annual Review of Biomedical Engineering, 2017, 19: 221-248. [4] 陈大千,张凡,郝鹏翼,等.结合多尺度通道注意力和边界增强的2D医学图像分割.计算机辅助设计与图形学报, 2022, 34(11): 1742-1752. (CHEN D Q, ZHANG F, HAO P Y, et al. 2D Medical Image Segmentation Combining Multi-scale Channel Attention and Boundary Enhancement. Journal of Computer-Aided Design and Computer Graphics, 2022, 34(11): 1742-1752.) [5] CAMPELLO V M, GKONTRA P, IZQUIERDO C, et al. Multi-centre, Multi-vendor and Multi-disease Cardiac Segmentation: The M&Ms Challenge. IEEE Transactions on Medical Imaging, 2021, 40(12): 3543-3554. [6] RONNEBERGER O, FISCHER P, BROX T.U-Net: Convolutional Networks for Biomedical Image Segmentation // Proc of the International Conference on Medical Image Computing and Computer-Assi-sted Intervention. Berlin, Germany: Springer, 2015: 234-241. [7] 高程玲,叶海良,曹飞龙.基于三重交互关注网络的医学图像分割算法.模式识别与人工智能, 2021, 34(5): 398-406. (GAO C L, YE H L, CAO F L.Medical Image Segmentation via Triplet Interactive Attention Network. Pattern Recognition and Artificial Intelligence, 2021, 34(5): 398-406.) [8] NING Z Y, ZHONG S Z, FENG Q J, et al. SMU-Net: Saliency-Guided Morphology-Aware U-Net for Breast Lesion Segmentation in Ultrasound Image. IEEE Transactions on Medical Imaging, 2022, 41(2): 476-490. [9] ZHOU Z W, SIDDIQUEE M M R, et al. TAJBAKHSH N, et al. UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Ima-ge Segmentation. IEEE Transactions on Medical Imaging, 2020, 39(6): 1856-1867. [10] GU Z W, CHENG J, FU H Z, et al. CE-Net: Context Encoder Network for 2D Medical Image Segmentation. IEEE Transactions on Medical Imaging, 2019, 38(10): 2281-2292. [11] GAO C L, YE H L, CAO F L, et al. Multiscale Fused Network with Additive Channel-Spatial Attention for Image Segmentation.Knowledge-Based Systems, 2021, 214. DOI: 10.1016/j.knosys.2021.106754. [12] WEI J, HU Y W, ZHANG R M, et al. Shallow Attention Network for Polyp Segmentation // Proc of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2021: 699-708. [13] CHEN G P, LI L, DAI Y, et al. AAU-Net: An Adaptive Attention U-Net for Breast Lesions Segmentation in Ultrasound Images. IEEE Transactions on Medical Imaging, 2023, 42(5): 1289-1300. [14] RAHMAN M M, MARCULESCU R.Medical Image Segmentation via Cascaded Attention Decoding // Proc of the IEEE/CVF Winter Conference on Applications of Computer Vision. Washington, USA: IEEE, 2023: 6211-6221. [15] ZHOU T, ZHOU Y, HE K L, et al. Cross-Level Feature Aggregation Network for Polyp Segmentation. Pattern Recognition, 2023, 140. DOI: 10.1016/j.patcog.2023.109555. [16] FAN D P, JI G P, ZHOU T, et al. PraNet: Parallel Reverse Atten-tion Network for Polyp Segmentation // Proc of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2020: 263-273. [17] CAO F L, GAO C L, YE H L.A Novel Method for Image Segmentation: Two-Stage Decoding Network with Boundary Attention. International Journal of Machine Learning and Cybernetics, 2022, 13: 1461-1471. [18] LI Y, ZHANG Y, CUI W G, et al. Dual Encoder-Based Dynamic-Channel Graph Convolutional Network with Edge Enhancement for Retinal Vessel Segmentation. IEEE Transactions on Medical Imaging, 2022, 41(8): 1975-1989. [19] LU Y, CHEN Y R, ZHAO D B, et al. CNN-G: Convolutional Neu-ral Network Combined with Graph for Image Segmentation with Theoretical Analysis. IEEE Transactions on Cognitive and Developmental Systems, 2021, 13(3): 631-644. [20] SHIN S Y, LEE S, YUN I D, et al. Deep Vessel Segmentation by Learning Graphical Connectivity. Medical Image Analysis, 2019, 58. DOI: 10.1016/j.media.2019.101556. [21] MENG Y D, ZHANG H R, GAO D X, et al. BI-GCN: Boundary-Aware Input-Dependent Graph Convolution Network for Biomedical Image Segmentation[C/OL].[2023-10-20]. https://arxiv.org/pdf/2110.14775v2.pdf. [22] MENG Y D, ZHANG H R, ZHAO Y T, et al. Graph-Based Region and Boundary Aggregation for Biomedical Image Segmentation. IEEE Transactions on Medical Imaging, 2022, 41(3): 690-701. [23] WANG K, ZHANG X H, LU Y T, et al. CGRNet: Contour-Guided Graph Reasoning Network for Ambiguous Biomedical Image Segmentation. Biomedical Signal Processing and Control, 2022, 75. DOI: 10.1016/j.bspc.2022.103621. [24] LIU F L, HUA Z, LI J J, et al. MFBGR: Multi-scale Feature Boundary Graph Reasoning Network for Polyp Segmentation. Engineering Applications of Artificial Intelligence, 2023, 123. DOI: 10.1016/j.engappai.2023.106213. [25] GAO S H, CHENG M M, ZHAO K, et al. Res2Net: A New Multi-scale Backbone Architecture. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(2): 652-662. [26] KIPF T N, WELLING M.Semi-Supervised Classification with Graph Convolutional Networks[C/OL]. [2023-10-20].https://arxiv.org/pdf/1609.02907.pdf. [27] CHEN S H, FU Y.Progressively Guided Alternate Refinement Network for RGB-D Salient Object Detection // Proc of European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 520-538. [28] WEI J, WANG S H, HUANG Q M.F3Net: Fusion, Feedback and Focus for Salient Object Detection. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 12321-12328. [29] RAHNAMA J, HÜLLERMEIER E. Learning Tversky Similarity // Proc of the International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems. Berlin, Germany: Springer, 2020: 269-280. [30] BERNAL J, SÁNCHEZ F J, FERNÁNDEZ-ESPARRACH G, et al. WM-DOVA Maps for Accurate Polyp Highlighting in Colonoscopy: Validation vs. Saliency Maps from Physicians. Computerized Medical Imaging and Graphics, 2015, 43: 99-111. [31] BERNAL J, SÁNCHEZ J, VILARINO F. Towards Automatic Po-lyp Detection with a Polyp Appearance Model. Pattern Recognition, 2012, 45(9): 3166-3182. [32] YAP M H, PONS G, MARTI J, et al. Automated Breast Ultrasound Lesions Detection Using Convolutional Neural Networks. IEEE Journal of Biomedical and Health Informatics, 2018, 22(4): 1218-1226. [33] ABRAHAM N, KHAN N M.A Novel Focal Tversky Loss Function with Improved Attention U-Net for Lesion Segmentation // Proc of the 16th IEEE International Symposium on Biomedical Imaging. Washington, USA: IEEE, 2019: 683-687. [34] KINGMA D P, BA J L, Adam: A Method for Stochastic Optimization[C/OL]. [2023-10-20]. https://arxiv.org/pdf/1412.6980.pdf. [35] ZHAO R J, QIAN B Y, ZHANG X L, et al. Rethinking Dice Loss for Medical Image Segmentation // Proc of the IEEE International Conference on Data Mining. Washington, USA: IEEE, 2020: 851-860. [36] WANG R X, CHEN S Y, JI C J, et al. Boundary-Aware Context Neural Network for Medical Image Segmentation. Medical Image Analysis, 2022, 78. DOI: 10.1016/j.media.2022.102395. [37] WANG S T, YIN Y Q, WANG D J, et al. An Interpretable Deep Neural Network for Colorectal Polyp Diagnosis under Colonoscopy. Knowledge-Based Systems, 2021, 234. DOI: 10.1016/j.knosys.2021.107568.
2024, Vol. 37 
