基于结构变换补全的边缘纹理双特征聚合图像修复方法

doi:10.16451/j.cnki.issn1003-6059.202505002

Abstract
Figure/Table
References (31)
Related Citation (9)

Download: PDF (6268 KB) HTML (1 KB)
Export: BibTeX | EndNote (RIS)

Abstract

The deficiencies in restoring plausible edge structures and complete textures within missing regions still emerge in existing neural network-based approaches for image inpainting. To address these issues, a method for edge-texture dual feature aggregation for image inpainting via structural transformation completion(ETSTC) is proposed. First, a structure transform completer module integrating axial attention and contextual transformer is designed. The module is combined with a structure smoother module to further complement and optimize edge structures. Thus, both local edge details and global structural patterns are effectively captured while edge noise and artifacts are suppressed. Second, an edge-guided feature aligner and an edge-texture dual-feature aggregator are developed. Scaling and offset parameters are adaptively learned to effectively resolve scale and offset discrepancies in dynamic aggregation of edge structural features and texture features across different feature space levels, and thereby the image inpainting performance is improved. Finally, experiments on three datasets verify the feasibility and effectiveness of ETSTC.

Key words： Image Inpainting Edge Guidance Structure Completion Feature Space Dual Feature Aggregation

Received: 14 April 2025

ZTFLH:

TP391

Fund:

National Natural Science Foundation of China(No.52375178), Natural Science Foundation of Shanxi Province(No.202203021211206,202203021211189)

Corresponding Authors: ZHANG Rongguo, Ph.D., professor. His research interests include image processing, computer vision and pattern recognition.

About author:: WEN Yihao, Master student. His research interests include image processing and computer vision.
HU Jing, Ph.D., professor. Her research interests include image processing and pattern recognition.
WANG Lifang, Ph.D., associate profe?ssor. Her research interests include image processing and computer vision.
LIU Xiaojun, Ph.D., professor. Her research interests include modern design theories and methods, and pattern recognition.

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	ZHANG Rongguo
	WEN Yihao
	HU Jing
	WANG Lifang
	LIU Xiaojun

Cite this article:

ZHANG Rongguo,WEN Yihao,HU Jing等. Edge-Texture Dual Feature Aggregation for Image Inpainting via Structural Transformation Completion[J]. Pattern Recognition and Artificial Intelligence, 2025, 38(5): 397-411.

URL:

http://manu46.magtech.com.cn/Jweb_prai/EN/10.16451/j.cnki.issn1003-6059.202505002 OR http://manu46.magtech.com.cn/Jweb_prai/EN/Y2025/V38/I5/397

[1] YANG J, RUHAIYEM N I R. Review of Deep Learning-Based Image Inpainting Techniques. IEEE Access, 2024, 12: 138441-138482.
[2] WALI A, NASEER A, TAMOOR M, et al. Recent Progress in Di-gital Image Restoration Techniques: A Review. Digital Signal Processing, 2023, 141. DOI: 10.1016/j.dsp.2023.104187.
[3] XIANG H Y, ZOU Q, NAWAZ M A, et al. Deep Learning for Image Inpainting: A Survey. Pattern Recognition, 2023, 134. DOI: 10.1016/j.patcog.2022.109046.
[4] CHEN Y T, XIA R L, YANG K, et al. DNNAM: Image Inpainting Algorithm via Deep Neural Networks and Attention Mechanism. Applied Soft Computing, 2024, 154. DOI: 10.1016/j.asoc.2024.111392.
[5] YU X X, XU L, LI J, et al. MagConv: Mask-Guided Convolution for Image Inpainting. IEEE Transactions on Image Processing, 2023, 32: 4716-4727.
[6] WAN Z Y, ZHANG J B, CHEN D D, et al. High-Fidelity Pluralistic Image Completion with Transformers // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2021: 4672-4681.
[7] NAZERI K, NG E, JOSEPH T, et al. EdgeConnect: Structure Guided Image Inpainting Using Edge Prediction // Proc of the IEEE/CVF International Conference on Computer Vision Workshops. Washington, USA: IEEE, 2019: 3265-3274.
[8] TYLEI?EK R, ŠÁRA R. Spatial Pattern Templates for Recognition of Objects with Regular Structure // Proc of the 35th German Conference on Pattern Recognition. Berlin, Germany: Springer, 2013: 364-374.
[9] LIU Z W, LUO P, WANG X G, et al. Deep Learning Face Attri-butes in the Wild // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2015: 3730-3738.
[10] ZHOU B L, LAPEDRIZA A, KHOSLA A, et al. Places: A 10 Million Image Database for Scene Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(6): 1452-1464.
[11] SHEN J H, KANG S H, CHAN T F.Euler's Elastica and Curvature-Based Inpainting. SIAM Journal on Applied Mathematics, 2003, 63(2): 564-592.
[12] TSCHUMPERLÉ D, DERICHE R.Vector-Valued Image Regularization with PDEs: A Common Framework for Different Applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(4): 506-517.
[13] CRIMINISI A, PÉREZ P, TOYAMA K. Region Filling and Object Removal by Exemplar-Based Image Inpainting. IEEE Transactions on Image Processing, 2004, 13(9): 1200-1212.
[14] RU?I? T, PI?URICA A. Context-Aware Patch-Based Image Inpainting Using Markov Random Field Modeling. IEEE Transactions on Image Processing, 2015, 24(1): 444-456.
[15] PATHAK D, KRAHENBUHL P, DONAHUE J, et al. Context Encoders: Feature Learning by Inpainting // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 2536-2544.
[16] CHEN G, ZHANG G P, YANG Z G, et al. Multi-scale Patch-GAN with Edge Detection for Image Inpainting. Applied Intelligence, 2023, 53(4): 3917-3932.
[17] WANG Y F, GUO D S, ZHAO H R, et al. Image Inpainting via Multi-scale Adaptive Priors. Pattern Recognition, 2025, 162. DOI: 10.1016/j.patcog.2025.111410.
[18] 邵新茹.叶海良.杨冰.等. 基于三阶段生成网络的图像修复.模式识别与人工智能, 2022, 35(12): 1047-1063.
(SHAO X R, YE H L, YANG B, et al. Image Inpainting with a Three-Stage Generative Network. Pattern Recognition and Artificial Intelligence, 2022, 35(12): 1047-1063.)
[19] GUO X F, YANG H Y, HUANG D.Image Inpainting via Conditional Texture and Structure Dual Generation // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2021: 14114-14123.
[20] LIU Q K, JIANG Y Q, TAN Z T, et al. Transformer Based Plura-listic Image Completion with Reduced Information Loss. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(10): 6652-6668.
[21] WANG Q, WANG Z C, ZHANG X P, et al. Art Image Inpainting with Style-Guided Dual-Branch Inpainting Network. IEEE Transactions on Multimedia, 2024, 26: 8026-8037.
[22] LI W B, LIN Z, ZHOU K, et al. MAT: Mask-Aware Transformer for Large Hole Image Inpainting // Proc of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 10748-10758.
[23] KO K, KIM C S.Continuously Masked Transformer for Image Inpainting // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2023: 13123-13132.
[24] ZHANG C, YANG W X, LI X, et al. MMGInpainting: Multi-modality Guided Image Inpainting Based on Diffusion Models. IEEE Transactions on Multimedia, 2024, 26: 8811-8823.
[25] HUAN L X, XUE N, ZHENG X W, et al. Unmixing Convolutional Features for Crisp Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(10): 6602-6609.
[26] CAO C J, FU Y W.Learning a Sketch Tensor Space for Image Inpainting of Man-Made Scenes // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2021: 14489-14498.
[27] WANG H Y, ZHU Y K, GREEN B, et al. Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 108-126.
[28] LI Y H, YAO T, PAN Y W, et al. Contextual Transformer Networks for Visual Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(2): 1489-1500.
[29] KOLODOCHKA D, POLYAKOVA S M, NESTERIUK P D O, et al. LaMa Network Architecture Search for Image Inpainting[C/OL].[2025-03-17]. https://ceur-ws.org/Vol-3790/paper32.pdf.
[30] HU J, SHEN L, SUN G.Squeeze-and-Excitation Networks // Proc of the IEEE/CVF International Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 7132-7141.
[31] PARK T, LIU M Y, WANG T C, et al. Semantic Image Synthesis with Spatially-Adaptive Normalization // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 2332-2341.