Abstract To address the problem of difficult segmentation and many voids in the transition regions of normal and abnormal tissues in liver histopathology images segmentation, a semantic segmentation network of pathological images of liver tissue based on multi-scale feature and attention mechanism is proposed. The fused multi-scale features are extracted in the encoder to improve the segmentation of the transition regions between normal and abnormal tissues. The attention mechanism is employed to model the correlation between spatial dimension and channel dimension to obtain the response of each pixel within each class as well as the dependency between channels, and the impact of many voids in liver histopathology images on the network learning is alleviated. Experiments demonstrate that the proposed network can segment the damaged regions of liver histopathology images more quickly and accurately.
Fund:Major Scientific Research Instrument Development Project of National Natural Science Foundation of China(No.81727802), Science and Technology Planning Project of Yulin City of Shaanxi Province(No.CXY-2020-017), Key Issues in Medical Image Analysis with the Fusion of Vision Perception and Cognition(No.62073260)
Corresponding Authors:
BU Qirong, Ph.D., associate professor. His research interests in-clude computer vision, medical image proce-ssing and machine learning.
About author:: ZHANG Aoqi, master student. Her re-search interests include medical image proce-ssing, machine learning and deep learning. KANG Yuxin, Ph.D. candidate. His re-search interests include medical image proce-ssing, machine learning and deep learning. WU Zhuoyue, master student. His research interests include computer vision and machine learning. CUI Lei, Ph.D., lecturer. His research in-terests include computer vision, medical image processing and deep learning.
ZHANG Aoqi,KANG Yuxin,WU Zhuoyue等. Semantic Segmentation Network of Pathological Images of Liver Tissue Based on Multi-scale Feature and Attention Mechanism[J]. , 2021, 34(4): 375-384.
[1] CAMPANELLA G, HANNA M G, GENESLAW L, et al. Clinical-Grade Computational Pathology Using Weakly Supervised Deep Learning on Whole Slide Images. Nature Medicine, 2019, 25: 1301-1309. [2] XING F Y, XIE Y P, SU H, et al. Deep Learning in Microscopy Image Analysis: A Survey. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(10): 4550-4568. [3] 崔 磊.病理图像精细化分析算法研究.博士学位论文.西安:西北大学, 2019. (CUI L. Research on Refined Analysis Algorithm of Pathological Image. Ph.D. Dissertation. Xi′an, China: Northwest University, 2019.) [4] 王飞跃,苟 超,王建功,等.平行皮肤:基于视觉的皮肤病分析框架.模式识别与人工智能, 2019, 32(7): 577-588. (WANG Y F, GOU C, WANG J G, et al. Parallel Skin: A Vision-Based Dermatological Analysis Framework. Pattern Recognition and Artificial Intelligence, 2019, 32(7): 577-588.) [5] 张 梅,陈 鸰,王飞跃,等.平行胃肠:基于ACP的智能胃肠疾病诊疗.模式识别与人工智能, 2019, 32(12): 1061-1071. (ZHANG M, CHEN G, WANG Y F, et al. Parallel Gastrointes-tine: An ACP-Based Approach for Intelligent Operations. Pattern Recognition and Artificial Intelligence, 2019, 32(12): 1061-1071.) [6] IOANNOU N, STANISAVLJEVIC M, ANGHEL A, et al. Accele-rated ML-Assisted Tumor Detection in High-Resolution Histopathology Images // Proc of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2019: 406-414. [7] TOKUNAGA H, TERAMOTO Y, YOSHIZAWA A, et al. Adaptive Weighting Multi-field-of-View CNN for Semantic Segmentation in Pathology // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 12597-12606. [8] QU H, YAN Z N, RIEDLINGER G M, et al. Improving Nuclei/Gland Instance Segmentation in Histopathology Images by Full Reso-lution Neural Network and Spatial Constrained Loss // Proc of the 22nd International Conference on Medical Image Computing and Compu-ter-Assisted Intervention. Berlin, Germany: Springer, 2019: 378-386. [9] ZHANG D H, SONG Y, LIU D N, et al. Panoptic Segmentation with an End-to-End Cell R-CNN for Pathology Image Analysis // Proc of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2018: 237-244. [10] OKADA T, SHIMADA R, SATO Y, et al. Automated Segmenta-tion of the Liver from 3D CT Images Using Probabilistic Atlas and Multi-level Statistical Shape Model // Proc of the 10th International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2007: 86-93. [11] CRUZ-GOMEZ C, WIEDERHOLD P, GUDINO-ZAYAS M. Auto-matic Liver Tissue Segmentation in Microscopic Images Using Fusion Color Space and Multiscale Morphological Reconstruction // Proc of the International Conference on Technological Advances in Electrical, Electronics and Computer Engineering. Washington, USA: IEEE, 2013: 88-92. [12] ROJAS-MORALEDA R, XIONG W, HALAMA N, et al. Robust Detection and Segmentation of Cell Nuclei in Biomedical Images Based on a Computational Topology Framework. Medical Image Analysis, 2017, 38: 90-103. [13] ROY M, WANG F S, TEODORO G, et al. Segmentation of Over-lapped Steatosis in Whole-Slide Liver Histopathology Microscopy Images // Proc of the 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Washington, USA: IEEE, 2018: 810-813. [14] XU Z Y. Computational Models for Automated Histopathological Assessment of Colorectal Liver Metastasis Progression. Ph.D. Di-ssertation. London, UK: Queen Mary University of London, 2019. [15] PAUL A, MUKHERJEE D P. Gland Segmentation from Histology Images Using Informative Morphological Scale Space // Proc of the IEEE International Conference on Image Processing. Washington, USA: IEEE, 2016: 4121-4125. [16] YANG L, ZHANG Y Z, CHEN J X, et al. Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmen-tation // Proc of the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2017: 399-407. [17] HE K M, ZHANG X Y, REN S Q, et al. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2014: 346-361. [18] WANG X L, GIRSHICK R, GUPTA A, et al. Non-local Neural Networks // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 7794-7803. [19] FU J, LIU J, TIAN H J, et al. Dual Attention Network for Scene Segmentation // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 3141-3149. [20] RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolu-tional Networks for Biomedical Image Segmentation // Proc of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2015: 234-241. [21] CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking Atrous Convolution for Semantic Image Segmentation[J/OL]. [2020-05-25]. https://arxiv.org/pdf/1706.05587.pdf. [22] IOFFE S, SZEGEDY C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[J/OL]. [2020-05-25]. https://arxiv.org/pdf/1502.03167.pdf. [23] WU Y X, HE K M. Group Normalization. International Journal of Computer Vision, 2020, 128: 742-755. [24] GRAHAM S, CHEN H, GAMPER J, et al. MILD-Net: Minimal Information Loss Dilated Network for Gland Instance Segmentation in Colon Histology Images. Medical Image Analysis, 2019, 52: 199-211. [25] ZHU Y H, CHEN Z N, ZHAO S, et al. ACE-Net: Biomedical Image Segmentation with Augmented Contracting and Expansive Paths // Proc of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2019: 712-720. [26] GLOROT X, BENGIO Y. Understanding the Difficulty of Training Deep Feedforward Neural Networks // Proc of the 13th Internatio-nal Conference on Artificial Intelligence and Statistics. New York, USA: ACM, 2010: 249-256. [27] SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Loca-lization. International Journal of Computer Vision, 2020, 128: 336-359. [28] OKTAY O, SCHLEMPER J, FOLGOC L L, et al. Attention U-Net: Learning Where to Look for the Pancreas[J/OL]. [2020-05-25]. https://arxiv.org/pdf/1804.03999.pdf. [29] ALOM M Z, HASAN M, YAKOPCIC C, et al. Recurrent Residual Convolutional Neural Network Based on U-Net(R2U-Net) for Medi-cal Image Segmentation[J/OL]. [2020-05-25]. https://arxiv.org/ftp/arxiv/papers/1802/1802.06955.pdf. [30] ZHOU Z W, SIDDIQUEE M M R, TAJBAKHSH N, et al. UNet++: A Nested U-Net Architecture for Medical Image Segmentation // Proc of the International Workshop on Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Berlin, Germany: Springer, 2018: 3-11. [31] CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-Deco-der with Atrous Separable Convolution for Semantic Image Segmen-tation // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 833-851. [32] HE K M, ZHANG X Y, REN S Q, et al. Deep Residual Learning for Image Recognition // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 770-778. [33] CHOLLET F. Xception: Deep Learning with Depthwise Separable Convolutions // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 1251-1258.
2021, Vol. 34 
