Multimodal Corpus Construction Based on Medical Image Segmentation Algorithm
LIN Yuping1, ZHENG Yaoyue2, ZHENG Haojie2, ZHANG Dong3, WANG Cong2, LI Xiaomian1, LI Yingyu1, TIAN Zhiqiang2
1. School of Foreign Studies, Xi′an Jiaotong University, Xi′an 710049 2. School of Software Engineering, Xi′an Jiaotong University, Xi′an 710049 3. School of Automation Science and Engineering, Xi′an Jiaotong University, Xi′an 710049
Abstract Electronic medical records(EMRs) corpus provides qualitative diagnosis results of related medical images. However, the good management of medical data may be affected due to the lacking of labeled images and texts and it is hard for medical students to acquire the related medical knowledge independently. To solve this problem, a medical image segmentation method based on the deep level set algorithm is proposed to segment medical images automatically and output contour results of the interested area and related quantitative indicators. Electronic medical record text is annotated grounded on natural language processing methods. The information representation of medical record texts and images of multimodal corpus is enhanced. Experimental results on the glaucoma image dataset show that the proposed method segments the optic disc and the optic cup in the fundus image accurately and a multimodal corpus with self-evident labeled images and EMRs is constructed effectively as well.
Fund:2017 Project in the 13th Five-Year Plan of Education Science of Shaanxi Province(No.SGH17H003)
Corresponding Authors:
TIAN Zhiqiang, Ph.D., associate professor. His research interests include machine vision, machine lear-ning and medical image processing.
About author:: LIN Yuping, master, associate professor. Her research interests include corpus constru-ction, second language acquisition. ZHENG Yaoyue, master student. Her research interests include machine learning and medical image analysis. ZHENG Haojie, master student. Her research interest includes image segmentation. ZHANG Dong, Ph.D. candidate. His research interests include machine vision, machine learning and medical image proce-ssing. WANG Cong, master student. Her research interests include image processing and video analysis. LI Xiaomian, master, lecturer. Her research interests include linguistics and medical English. LI Yingyu, Ph.D., associate professor. Her research interests include corpus lingui-stics, translation studies and TESOL.
[1] 杨锦锋,于秋滨,关 毅,等.电子病历命名实体识别和实体关系抽取研究综述.自动化学报, 2014, 40(8): 1537-1562. (YANG J F, YU Q B, GUAN Y, et al. An Overview of Research on Electronic Medical Record Oriented Named Entity Recognition and Entity Relation Extraction. Acta Automatica Sinica, 2014, 40(8): 1537-1562.) [2] 晏归来,安新颖,范少萍,等.国外生物医学文本语料库分类及特点研究.医学信息学杂志, 2018, 39(10): 74-80. (YAN G L, AN X Y, FAN S P, et al. Study on the Categories and Characteristics of Overseas Biomedical Text Corpuses. Journal of Medical Informatics, 2018, 39(10): 74-80.) [3] 简 哲,李 燕.电子病历自然语言处理测评发展.医学信息学杂志, 2016, 37(12): 10-13, 21. (JIAN Z, LI Y. The Development of EMR Natural Language Processing Evaluation. Journal of Medical Informatics, 2016, 37(12): 10-13, 21.) [4] LITJENS G, KOOI T, BEJNORDI B E, et al. A Survey on Deep Learning in Medical Image Analysis. Medical Image Analysis, 2017, 42: 60-88. [5] 苏 嘉,何 彬,吴 昊,等.基于中文电子病历的心血管疾病风险因素标注体系及语料库构建.自动化学报, 2019, 45(2): 420-426. (SU J, HE B, WU H, et al. Annotation Scheme and Corpus Construction for Cardiovascular Diseases Risk Factors From Chinese Electronic Medical Records. Acta Automatica Sinica, 2019, 45(2): 420-426.) [6] LI P L, YUAN Z M, TU W, et al. Medical Knowledge Extraction and Analysis from Electronic Medical Records Using Deep Learning. Chinese Medical Sciences Journal, 2019, 34(2): 133-139. [7] 蒋志鹏,关 毅.面向中文电子病历的句法分析融合模型.自动化学报, 2019, 45(2): 276-288. (JIANG Z P, GUAN Y. A Fusion Model for Chinese Electronic Medical Record Parsing. Acta Automatica Sinica, 2019, 45(2): 276-288) [8] 魏 微.医学英语多模态语料库系统构建以及应用.微型电脑应用, 2020, 36(2): 75-78. (WEI W. Construction and Application of Medical English Multimodal Corpus System. Microcomputer Applications, 2020, 36(2): 75-78.) [9] 张成智.国内外多模态语料库建设现状.长江丛刊, 2018(31): 56. (ZHANG C Z. Current Situation of Multi-Modal Corpuses Construction at Domestic and Overseas. Yangtze River Series, 2018(31): 56.) [10] THAKUR N, JUNEJA M. Survey on Segmentation and Classification Approaches of Optic Cup and Optic Disc for Diagnosis of Glaucoma. Biomedical Signal Processing and Control, 2018, 42: 162-189. [11] ALMAZROA A, BURMAN R, RAAHEMIFAR K, et al. Optic Disc and Optic Cup Segmentation Methodologies for Glaucoma Image Detection: A Survey. Journal of Ophthalmology, 2015. DOI: 10.1155/2015/180972. [12] HALEEM M S, HAN L X, VAN HEMERT J, et al. Automatic Extraction of Retinal Features from Colour Retinal Images for Glaucoma Diagnosis: A Review. Computerized Medical Imaging and Graphics, 2013, 37(7/8): 581-596. [13] WONG D W K, LIU J, LIM J H, et al. Level-Set Based Automatic Cup-To-Disc Ratio Determination Using Retinal Fundus Images in ARGALI // Proc of the 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Washington, USA: IEEE, 2008: 2266-2269. [14] ZHENG Y J, STAMBOLIAN D, O′BRIEN J, et al. Optic Disc and Cup Segmentation from Color Fundus Photograph Using Graph Cut with Priors // Proc of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2013: 75-82. [15] DAI B S, WU X Q, BU W. Optic Disc Segmentation Based on Variational Model with Multiple Energies. Pattern Recognition, 2017, 64: 226-235. [16] JOSHI G D, SIVASWAMY J, KRISHNADAS S R. Optic Disk and Cup Segmentation from Monocular Color Retinal Images for Glaucoma Assessment. IEEE Transactions on Medical Imaging, 2011, 30(6): 1192-1205. [17] ZHAO H S, SHI J P, QI X J, et al. Pyramid Scene Parsing Network // Proc of the IEEE Conference on Computer Vision and Pa-ttern Recognition. Washington, USA: IEEE, 2017: 6230-6239. [18] LONG J, SHELHAMER E, DARRELL T. Fully Convolutional Networks for Semantic Segmentation // Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 3431-3440. [19] RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional Networks for Biomedical Image Segmentation // Proc of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2015: 234-241. [20] CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-Deco-der with Atrous Separable Convolution for Semantic Image Segmentation // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 833-851. [21] MANINIS K K, PONT-TUSET J, ARBELÁEZ P, et al. Deep Re-tinal Image Understanding // Proc of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2016: 140-148. [22] SIMONYAN K, ZISSERMAN A. Very Deep Convolutional Networks for Large-Scale Image Recognition[C/OL]. [2020-06-12]. https://arxiv.org/pdf/1409.1556.pdf. [23] WOO S, PARK J, LEE J Y, et al. CBAM: Convolutional Block Attention Module // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 3-19. [24] HE K M, ZHANG X Y, REN S Q, et al. Deep Residual Learning for Image Recognition // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 770-778. [25] WANG Z A, ACUNA D, LING H, et al. Object Instance Annotation with Deep Extreme Level Set Evolution // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 7492-7500. [26] KHAN S, SHAHIN A H, VILLAFRUELA J, et al. Extreme Points Derived Confidence Map as a Cue for Class-Agnostic Segmentation Using Deep Neural Network // Proc of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2019: 66-73. [27] CHEN X, WILLIAMS B M, VALLABHANENI S R, et al. Lear-ning Active Contour Models for Medical Image Segmentation // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 11624-11632. [28] THAM Y C, LI X, WONG T Y, et al. Global Prevalence of Glaucoma and Projections of Glaucoma Burden through 2040: A Syste-matic Review and Meta-Analysis. Ophthalmology, 2014, 121(11): 2081-2090. [29] YOHANNAN J, BOLAND M V. The Evolving Role of the Relationship between Optic Nerve Structure and Function in Glaucoma. Ophthalmology, 2017, 124(12): S66-S70. [30] LI Z X, HE Y F, KEEL S, et al. Efficacy of a Deep Learning System for Detecting Glaucomatous Optic Neuropathy Based on Color Fundus Photographs. Ophthalmology, 2018, 125(8): 1199-1206. [31] ORLANDO J I, FU H Z, BREDA J B, et al. Refuge Challenge: A Unified Framework for Evaluating Automated Methods for Glaucoma Assessment from Fundus Photographs. Medical Image Analysis, 2020, 59. DOI: 10.1016/j.media.2019.101570. [32] MANINIS K K, CAELLES S, PONT-TUSET J, et al. Deep Extreme Cut: From Extreme Points to Object Segmentation // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 616-625.
2021, Vol. 34 
