Thyroid Nodule Segmentation Model Integrating Global Reasoning and MLP Architecture
LI Binrong1, XIE Jun1, LI Gang2, XU Xinying3, LAN Zijun1
1.College of Information and Computer, Taiyuan University of Technology, Jinzhong 030600; 2.College of Software, Taiyuan University of Technology, Jin-zhong 030600; 3.College of Electrical and Power Engineering, Taiyuan University of Technology, Taiyuan 030024
Abstract To address the problems of large noise interference in ultrasound images and variable nodule size and high computational complexity of the existing thyroid nodule segmentation methods, a segmentation model combining global reasoning and multi-layer perception(MLP) architecture is proposed. The model is based on the axial shift MLP module, and hence the interaction between different spatial location features is realized with less computational complexity. The end-to-end global reasoning unit is integrated into the encoder and the global information interaction is conducted based on graph convolutional networks to alleviate the interference of image noise. The pyramid feature layer is introduced into the decoder and multi-scale feature interaction is performed to deal with the problem of variable nodule size. Experimental results on DDIT datasets show that the proposed model yields better performance, and it can be applied to other medical image segmentation task, such as breast nodule segmentation and retinal vessel segmentation.
Fund:Supported by Natural Science Foundation of Shanxi Province(No.201901D111091)
Corresponding Authors:XIE Jun, Ph.D., associate professor. Her research interests include image semantic segmentation and intelligent information processing.
About author:: About Author:LI Binrong, master student. Her research interests include computer vision and medical image processing.
LI Gang, Ph.D., associate professor. His research interests include artificial intelligence and visual information processing.
XU Xinying, Ph.D., professor. His research interests include computer vision and intelligent control.
LAN Zijun, master student. His research interests include deep learning and medical image processing.
LI Binrong,XIE Jun,LI Gang等. Thyroid Nodule Segmentation Model Integrating Global Reasoning and MLP Architecture[J]. Pattern Recognition and Artificial Intelligence, 2022, 35(7): 649-660.
[1] 左东奇,韩霖,陈科,等.基于卷积神经网络提取超声图像甲状腺结节钙化点的研究.生物医学工程学杂志, 2018, 35(5): 679-687. (ZUO D Q, HAN L, CHEN K, et al. Extraction of Calcification in Ultrasonic Images Based on Convolution Neural Network. Journal of Biomedical Engineering, 2018, 35(5): 679-687.) [2] ZHAO H S, SHI J P, QI X J, et al. Pyramid Scene Parsing Network // Proc of the IEEE Conference on Computer Vision and Pa-ttern Recognition. Washington, USA: IEEE, 2017: 6230-6239. [3] CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 833-851. [4] LIU M Y, YUAN X G, ZHANG Y A, et al. An End to End Thyroid Nodule Segmentation Model Based on Optimized U-Net Convolutional Neural Network // Proc of the International Symposium on Artificial Intelligence in Medical Sciences. New York, USA: ACM, 2020: 74-78. [5] TOLSTIKHIN I, HOULSBY N, KOLESNIKOV A, et al. MLP-Mixer: An All-MLP Architecture for Vision[C/OL].[2021-11-06]. https://arxiv.org/pdf/2105.01601.pdf. [6] CHEN S F, XIE E Z, GE C J, et al. CycleMLP: A MLP-Like Architecture for Dense Prediction[C/OL].[2021-11-06]. https://arxiv.org/abs/2107.10224.pdf. [7] 高程玲,叶海良,曹飞龙.基于三重交互关注网络的医学图像分割算法. 模式识别与人工智能, 2021, 34(5): 398-406. (GAO C L, YE H L, CAO F L. Medical Image Segmentation via Triplet Interactive Attention Network. Pattern Recognition and Artificial Intelligence, 2021, 34(5): 398-406.) [8] WANG X L, GIRSHICK R, GUPTA A, et al. Non-local Neural Networks // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 7794-7803. [9] FU J, LIU J, TIAN H J, et al. Dual Attention Network for Scene Segmentation // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 3141-3149. [10] VASWANI A, SHAZEER N, PARMAR N, et al. Attention Is All You Need // Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2017: 6000-6010. [11] ZHANG S Y, YAN S P, HE X M. LatentGNN: Learning Efficient Non-local Relations for Visual Recognition // Proc of the 36th International Conference on Machine Learning. San Diego, USA: JMLR, 2019: 7374-7383. [12] LI Q M, HAN Z C, WU X M. Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning // Proc of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI, 2018: 3538-3545. [13] LIANG X D, HU Z T, ZHANG H, et al. Symbolic Graph Reaso-ning Meets Convolutions // Proc of the 32nd Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2018: 1853-1863. [14] LIAN D Z, YU Z H, SUN X, et al. AS-MLP: An Axial Shifted MLP Architecture for Vision[C/OL].[2021-11-06]. https://arxiv.org/pdf/2107.08391.pdf. [15] HUANG Y H, PROESMANS M, VAN GOOL L. Context-Aware Padding for Semantic Segmentation[C/OL].[2021-11-06].https://arxiv.org/pdf/2109.07854.pdf. [16] CHEN Y P, ROHRBACH M, YAN Z C, et al. Graph-Based Global Reasoning Networks // Proc of the 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 433-442. [17] 张墺琦,亢宇鑫,武卓越,等.基于多尺度特征和注意力机制的肝脏组织病理图像语义分割网络.模式识别与人工智能, 2021, 34(4): 375-384. (ZHANG A Q, KANG Y X, WU Z Y, et al. Semantic Segmentation Network of Pathological Images of Liver Tissue Based on the Multi-scale Feature and Attention Mechanism. Pattern Recognition and Artificial Intelligence, 2021, 34(4): 375-384.) [18] 黄鸿,王涛,李远,等. 基于深度特征融合的癌症病理图像分割网络.光子学报, 2022, 51(3): 1-12. (HUANG H, WANG T, LI Y, et al. Cancer Pathological Segmentation Network Based on Depth Feature Fusion. Acta Photonica Sinica, 2022, 51(3): 1-12.) [19] LI S H, SUI X C, LUO X D, et al. Medical Image Segmentation Using Squeeze-and-Expansion Transformers // Proc of the 30th International Joint Conference on Artificial Intelligence. San Francisco, USA: IJCAI, 2021: 807-815. [20] PEDRAZA L, VARGAS C, NARVÁEZ F, et al.. An Open Access Thyroid Ultrasound Image Database // Proc of the 10th Internatio-nal Symposium on Medical Information Processing and Analysis. Bellingham, USA: SPIE, 2015. DOI: 10.1117/12.2073532. [21] YU F, KOLTUN V. Multi-scale Context Aggregation by Dilated Convolutions[C/OL]. [2021-11-06].https://arxiv.org/pdf/1511.07122.pdf. [22] MOSHKOV N, MATHE B, KERTESZ-FARKAS A, et al. Test-Time Augmentation for Deep Learning-Based Cell Segmentation on Microscopy Images. Scientific Reports, 2020, 10. DOI: 10.1038/s41598-020-61808-3. [23] CHAURASIA A, CULURCIELLO E. LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation[C/OL]. [2021-11-06].https://arxiv.org/pdf/1707.03718.pdf. [24] LI H C, XIONG P F, AN J, et al. Pyramid Attention Network for Semantic Segmentation[C/OL].[2021-11-06]. https://arxiv.org/pdf/1805.10180.pdf. [25] RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutio-nal Networks for Biomedical Image Segmentation // Proc of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2015: 234-241. [26] ZHOU Z W, SIDDIQUEE M M R, TAJBAKHSH N, et al. UNet++: A Nested U-Net Architecture for Medical Image Segmentation // Proc of the 4th International Workshop on Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Berlin, Germany: Springer, 2018: 3-11. [27] WANG M Y, YUAN C L, WU D S, et al. Automatic Segmentation and Classification of Thyroid Nodules in Ultrasound Images with Convolutional Neural Networks // Proc of the International Confe-rence on Segmentation, Classification, and Registration of Multi-modality Medical Imaging Data. Berlin, Germany: Springer, 2020: 109-115. [28] AL-DHABYANI W, GOMAA M, KHALED H, et al. Dataset of Breast Ultrasound Images. Data in Brief, 2019, 28. DOI: 10.1016/j.dib.2019.104863. [29] STAAL J, ABRÀMOFF M D, NIEMEIJER M, et al.Ridge-Based Vessel Segmentation in Color Images of the Retina. IEEE Transactions on Medical Imaging, 2004, 23(4): 501-509.
2022, Vol. 35 
