解耦表征学习视角下认知图像属性特征的图像生成方法

doi:10.16451/j.cnki.issn1003-6059.202407006

摘要
图/表
参考文献
相关文章 (2)

全文: PDF (4136 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要在生成式人工智能领域,解耦表征学习的研究进一步推动图像生成方法的发展,但现有的解耦方法更多地关注图像生成的低维表示,忽略目标变化图像内在的可解释因素,导致生成的图像容易受到其它不相关属性特征的影响.为此,文中提出解耦表征学习视角下认知图像属性特征的图像生成方法.首先,从生成模型的潜在空间出发,通过训练获得关于目标变化图像的候选遍历方向.然后,构建无监督语义分解策略,并基于候选遍历的方向联合发现嵌入在潜在空间中的可解释方向.最后,利用解耦编码器和对比学习构建对比模拟器和变化空间,进而由可解释方向提取目标变化图像的解耦表征并生成图像.在5个解耦数据集上的实验表明文中方法性能较优.

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	蔡江海
	黄成泉
	王顺霞
	罗森艳
	杨贵燕
	周丽华

关键词 ：解耦表征学习, 潜在空间, 可解释方向, 图像生成, 变化空间

Abstract：In the field of generative artificial intelligence, the research of disentangled representation learning further promotes the development of image generation methods. However, existing disentanglement methods pay more attention to low-dimensional representation of image generation, ignoring inherent interpretable factors of the target variation image. This oversight results in generated image being susceptible to the influence of other irrelevant attribute features. To address this issue, an image generation method for cognizing image attribute features from the perspective of disentangled representation learning is proposed. Firstly, candidate traversal directions for the target variation image are obtained by training, starting from the latent space of the generative model. Secondly, an unsupervised semantic decomposition strategy is constructed, and the interpretable directions embedded in the latent space are jointly discovered based on the direction of candidate traversals. Finally, a contrast simulator and a variation space are constructed using disentangled encoders and contrastive learning. Consequently, the disentangled representations of the target variation image are extracted from the interpretable directions and the image is generated. Extensive experiments on five popular disentanglement datasets demonstrate the superior performance of the proposed method.

Key words： Disentangled Representation Learning Latent Space Interpretable Direction Image Ge-neration Variation Space

收稿日期: 2024-04-26

ZTFLH:

TP391

基金资助:国家自然科学基金项目(No.62062024)、贵州省科技计划项目(No.黔科合基础-ZK[2021]一般342)、贵州省研究生教育教学改革重点项目(No.黔教合YJSJGKT [2021]018)、贵州省教育厅自然科学研究项目(No.黔教技[2022]015)、贵州省模式识别与智能系统重点实验室2022年度开放课题(No.GZMUKL[2022]KF03)资助

通讯作者: 黄成泉,博士,教授,主要研究方向为深度学习、图像处理.E-mail:hcq@gzmu.edu.cn.

作者简介: 蔡江海,硕士研究生,主要研究方向为深度学习、图像处理、解耦表征学习.E-mail:870152989@qq.com.王顺霞,硕士研究生,主要研究方向为机器学习、模式识别.E-mail:2689826749@qq.com.罗森艳,硕士研究生,主要研究方向为机器学习、模式识别.E-mail:1563770769@qq.com.杨贵燕,硕士研究生,主要研究方向为机器学习、模式识别.E-mail:2393350042@qq.com.周丽华,硕士,副教授,主要研究方向为深度学习、图像处理.E-mail:zlh@gzmu.edu.cn.

引用本文:

蔡江海, 黄成泉, 王顺霞, 罗森艳, 杨贵燕, 周丽华. 解耦表征学习视角下认知图像属性特征的图像生成方法[J]. 模式识别与人工智能, 2024, 37(7): 638-651. CAI Jianghai, HUANG Chengquan, WANG Shunxia, LUO Senyan, YANG Guiyan, ZHOU Lihua. Image Generation Method for Cognizing Image Attribute Features from the Perspective of Disentangled Representation Learning. Pattern Recognition and Artificial Intelligence, 2024, 37(7): 638-651.

链接本文:

http://manu46.magtech.com.cn/Jweb_prai/CN/10.16451/j.cnki.issn1003-6059.202407006 或 http://manu46.magtech.com.cn/Jweb_prai/CN/Y2024/V37/I7/638

[1] ABDAL R, QIN Y P, WONKA P. Image2StyleGAN: How to Embed Images into the StyleGAN Latent Space? // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 4431-4440.
[2] PARK T, ZHU J Y, WANG O, et al. Swap Autoencoder for Deep Image Manipulation // Proc of the 34th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2020: 7198-7211.
[3] YANG W L, HUANG J D, LUO D C, et al. Efficient Disentangled Representation Learning for Multi-modal Finger Biometrics. Pattern Recognition, 2024, 145. DOI: 10.1016/j.patcog.2023.109944.
[4] ZHANG C X, WANG C, ZHAO Y F, et al. DR²: Disentangled Recurrent Representation Learning for Data-Efficient Speech Video Synthesis // Proc of the IEEE/CVF Winter Conference on Applications of Computer Vision. Washington, USA: IEEE, 2024: 6192-6202.
[5] HIGGINS I, MATTHEY L, PAL A, et al. β-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework[C/OL].[2024-03-19]. https://openreview.net/pdf?id=Sy2fzU9gl.
[6] CHEN X, DUAN Y, HOUTHOOFT R, et al. InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets // Proc of the 30th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2016: 2180-2188.
[7] ZHU X Q, XU C, TAO D C. Where and What? Examining Interpretable Disentangled Representations // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 5857-5866.
[8] KAZEMI H, IRANMANESH S M, NASRABADI N. Style and Content Disentanglement in Generative Adversarial Networks // Proc of the IEEE Winter Conference on Applications of Computer Vision. Washington, USA: IEEE, 2019: 848-856.
[9] KARRAS T, LAINE S, AILA T. A Style-Based Generator Architecture for Generative Adversarial Networks // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 4396-4405.
[10] REN X C, YANG T, WANG Y W, et al. Learning Disentangled Representation by Exploiting Pretrained Generative Models: A Contrastive Learning View[C/OL].[2024-03-19]. https://arxiv.org/pdf/2102.10543.pdf.
[11] GOETSCHALCKX L, ANDONIAN A, OLIVA A, et al. GANalyze: Toward Visual Definitions of Cognitive Image Properties // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 5743-5752.
[12] HÄRKÖNEN E, HERTZMANN A, LEHTINEN J, et al. GANspace: Discovering Interpretable GAN Controls // Proc of the 34th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2020: 9841-9850.
[13] SHEN Y J, YANG C Y, TANG X O, et al. InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 44(4): 2004-2018.
[14] SHEN Y J, ZHOU B L. Closed-Form Factorization of Latent Semantics in GANs // Proc of the IEEE/CVF Conference on Compu-ter Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 1532-1540.
[15] 李雅琪,王杰,王锋,等. 基于层次对比学习的半监督节点分类算法.模式识别与人工智能, 2023, 36(8): 712-720.
(LI Y Q, WANG J, WANG F, et al. Semi-Supervised Node Cla-ssification Algorithm Based on Hierarchical Contrastive Learning. Pattern Recognition and Artificial Intelligence, 2023, 36(8): 712-720.)
[16] TAO C X, ZHU X Z, SU W J, et al. Siamese Image Modeling for Self-Supervised Vision Representation Learning // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2023: 2132-2141.
[17] WANG X, QI G J. Contrastive Learning with Stronger Augmentations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(5): 5549-5560.
[18] 许睿,邵帅,曹维佳,等. 基于重构对比的广义零样本图像分类.模式识别与人工智能, 2022, 35(12): 1078-1088.
(XU R, SHAO S, CAO W J, et al. Generalized Zero-Shot Image Classification Based on Reconstruction Contrast. Pattern Recognition and Artificial Intelligence, 2022, 35(12):1078-1088.)
[19] LIU R, GE Y X, CHOI C L, et al. DivCo: Diverse Conditional Image Synthesis via Contrastive Generative Adversarial Network // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 16372-16381.
[20] HÉNAFF O J, SRINIVAS A, DE FAUW J, et al. Data-Efficient Image Recognition with Contrastive Predictive Coding // Proc of the 37th International Conference on Machine Learning. San Diego, USA: JMLR, 2020: 4182-4192.
[21] EASTWOOD C, WILLIAMS C K I. A Framework for the Quantitative Evaluation of Disentangled Representations[C/OL].[2024-03-19].https://openreview.net/pdf?id=By-7dz-AZ.
[22] CHEN R T Q C, LI X C, GROSSE R, et al. Isolating Sources of Disentanglement in Variational Autoencoders[C/OL].[2024-03-19]. https://arxiv.org/pdf/1802.04942.
[23] KIM H, MNIH A. Disentangling by Factorising. Proceedings of Machine Learning Research, 2018, 80: 2649-2658.
[24] LEEB F, LANZILLOTTA G, ANNADANI Y, et al. Structure by Architecture: Structured Representations without Regularization[C/OL].[2024-03-19]. https://openreview.net/pdf?id=O_lFCPaF48t.
[25] YANG T, WANG Y W, LAN C L, et al. Vector-Based Representation Is the Key: A Study on Disentanglement and Compositional Generalization[C/OL].[2024-03-19]. https://arxiv.org/pdf/2305.18063.
[26] ESTERMANN B, WATTENHOFER R. DAVA: Disentangling Ad-versarial Variational Autoencoder[C/OL]. [2024-03-19].https://arxiv.org/pdf/2303.01384.pdf.
[27] YANG T, WANG Y W, LU Y, et al. Visual Concepts Tokenization // Proc of the 36th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2022: 31571-31582.
[28] PREECHAKUL K, CHATTHEE N, WIZADWONGSA S, et al. Di-ffusion Autoencoders: Toward a Meaningful and Decodable Representation // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 10609-10619.
[29] WU A C, ZHENG W S. Factorized Diffusion Autoencoder for Unsupervised Disentangled Representation Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 2024,38(6): 5930-5939.
[30] KHRULKOV V, MIRVAKHABOVA L, OSELEDETS I, et al. Dis-entangled Representations from Non-disentangled Models[C/OL].[2024-03-19]. https://arxiv.org/pdf/2102.06204.pdf.
[31] YANG T, WANG Y W, LÜ Y, et al. DisDiff: Unsupervised Disentanglement of Diffusion Probabilistic Models // Proc of the 37th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2023: 69130-69156.
[32] YANG T, REN X C, WANG Y W, et al. Towards Building a Group-Based Unsupervised Representation Disentanglement Framework[C/OL].[2024-03-19]. https://arxiv.org/pdf/2102.10303.