压缩引导双曲插值融合的视觉神经解码方法

doi:10.16451/j.cnki.issn1003-6059.202606003

摘要
图/表
参考文献(24)
相关文章 (1)

全文: PDF (5625 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要

基于脑电信号的视觉神经解码旨在从脑活动中检索受试者感知的图像.针对脑电信号信噪比低、视觉教师特征冗余及单一教师难以兼顾语义结构与感知细节等问题,文中提出压缩引导双曲插值融合的视觉神经解码方法(Compression-Guided Hyperbolic Interpolation Fusion Method for Visual Neural Decoding, CHIF).首先,在结构增强分支和外观保持分支上分别引入教师侧压缩适配器,对视觉表征进行压缩适配,降低高维视觉特征与EEG表征的模态差异.然后,设计双曲插值融合模块,将教师特征映射至双曲空间,通过测地线插值构造统一教师目标,并结合双曲距离和双向对比学习实现跨模态对齐.在10名被试EEG-图像检索数据集上的实验表明,CHIF取得有效的检索结果.

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	张剀凡
	刘军
	季毓淇
	何立火

关键词 ：脑电-图像检索, 视觉神经解码, 双曲插值融合, 跨模态对齐

Abstract：

Visual neural decoding based on electroencephalography(EEG) is designed to retrieve images perceived by subjects from brain activity. To address the issues of low signal-to-noise ratio in EEG signals, redundancy in visual teacher features and the difficulty for a single teacher to simultaneously preserve both semantic structure and perceptual details, a compression-guided hyperbolic interpolation fusion method for visual neural decoding (CHIF) is proposed in this paper. First, lightweight compression modules are introduced into the structure-enhancement branch and the appearance-preservation branch to compress and adapt visual representations. Consequently, the modality gap between high-dimensional visual features and EEG representations is reduced. Second, a hyperbolic interpolation fusion module is designed. Teacher features are mapped into hyperbolic space. A unified teacher target is constructed through geodesic interpolation. Cross-modal alignment is achieved by combining hyperbolic distance and bidirectional contrastive learning. The experiments on an EEG-image retrieval dataset of 10 subjects demonstrate that CHIF achieves effective retrieval performance.

Key words： EEG-Based Image Retrieval Visual Neural Decoding Hyperbolic Interpolation Fusion Cross-Modal Alignment

收稿日期: 2026-05-11

ZTFLH:

TP391.4

基金资助:

国家自然科学基金项目(No.62276203)、新一代人工智能国家科技重大专项项目(No.2025ZD0123601)资助

通讯作者: 何立火,博士,教授,主要研究方向为信息感知、视觉计算、类脑智能.E-mail:lhhe@mail.xidian.edu.cn.

作者简介: 张剀凡,博士研究生,主要研究方向为视觉神经解码、人工智能.E-mail:kaifanzhang@stu.xidian.edu.cn.
刘军,博士研究生,主要研究方向为目标检测、缺陷检测.E-mail:26021110022@stu.xidian.edu.cn.
季毓淇,博士研究生,主要研究方向为目标检测、域自适应开集目标检测、功能检测.E-mail:24021110955@stu.xidian.edu.cn.
第二十八届中国科协年会学术论文

引用本文:

张剀凡, 刘军, 季毓淇, 何立火. 压缩引导双曲插值融合的视觉神经解码方法[J]. 模式识别与人工智能, 2026, 39(6): 504-518. ZHANG Kaifan, LIU Jun, JI Yuqi, HE Lihuo. Compression-Guided Hyperbolic Interpolation Fusion Method for Visual Neural Decoding. Pattern Recognition and Artificial Intelligence, 2026, 39(6): 504-518.

链接本文:

http://manu46.magtech.com.cn/Jweb_prai/CN/10.16451/j.cnki.issn1003-6059.202606003 或 http://manu46.magtech.com.cn/Jweb_prai/CN/Y2026/V39/I6/504

[1] DICARLO J J, ZOCCOLAN D, RUST N C.How Does the Brain Solve Visual Object Recognition? Neuron, 2012, 73(3): 415-434.
[2] 黄凯奇,谭铁牛.视觉认知计算模型综述.模式识别与人工智能, 2013, 26(10): 951-958.
(HUANG K Q, TAN T N.Review on Computational Model for Vision. Pattern Recognition and Artificial Intelligence, 2013, 26(10): 951-958.)
[3] KAY K N, NASELARIS T, PRENGER R J, et al. Identifying Na-tural Images from Human Brain Activity. Nature, 2008, 452(7185): 352-355.
[4] GIFFORD A T, DWIVEDI K, ROIG G, et al. A Large and Rich EEG Dataset for Modeling Human Visual Object Recognition. NeuroImage, 2022, 264. DOI: 10.1016/j.neuroimage.2022.119754.
[5] 《脑机接口关键科学问题、关键核心技术及其布局研究》项目组.脑机接口技术发展现状及未来展望.科学与社会, 2024, 14(3): 2-25.
(Project Team for the "Study of Key Scientific Issues, Key Core Technologies and Their Layout for Brain-Computer Interfaces". Brain-Computer Interface Technology: Current Development Status and Future Outlook. Science and Society, 2024, 14(3): 2-25.)
[6] SONG Y H, LIU B C, LI X, et al. Decoding Natural Images from EEG for Object Recognition[C/OL].[2026-04-16]. https://arxiv.org/pdf/2308.13234.
[7] DU C D, FU K C, LI J P, et al. Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(9): 10760-10777.
[8] SRINIVASAN R, WINTER W R, DING J, et al. EEG and MEG Coherence: Measures of Functional Connectivity at Distinct Spatial Scales of Neocortical Dynamics. Journal of Neuroscience Methods, 2007, 166(1): 41-52.
[9] LAWHERN V J, SOLON A J, WAYTOWICH N R, et al. EEGNet: A Compact Convolutional Neural Network for EEG-Based Brain-Computer Interfaces. Journal of Neural Engineering, 2018, 15(5). DOI: 10.1088/1741-2552/aace8c.
[10] WEI Y Y, CAO L, LI H, et al. MB2C: Multimodal Bidirectional Cycle Consistency for Learning Robust Visual Neural Representations // Proc of the 32nd ACM International Conference on Multimedia. New York, USA: ACM, 2024: 8992-9000.
[11] ZHANG K F, HE L H, JIANG X, et al. CognitionCapturer: Decoding Visual Stimuli from Human EEG Signal with Multimodal Information. Proceedings of the AAAI Conference on Artificial Intelligence, 2025, 39(13): 14486-14493.
[12] WU H T, LI Q, ZHANG C Q, et al. Bridging the Vision-Brain Gap with an Uncertainty-Aware Blur Prior // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2025: 2246-2257.
[13] WU L K, LI J, REN Z Q, et al. Shrinking the Teacher: An Adaptive Teaching Paradigm for Asymmetric EEG-Vision Alignment. Proceedings of the AAAI Conference on Artificial Intelligence, 2026, 40(21): 17859-17867.
[14] GAO P, GENG S J, ZHANG R R, et al. CLIP-Adapter: Better Vision-Language Models with Feature Adapters. International Journal of Computer Vision, 2024, 132(2): 581-595.
[15] 曹洪涛,钟子平,陈远方,等.非侵入式脑机接口控制策略的研究进展.生物医学工程学杂志, 2022, 39(5): 1033-1040.
(CAO H T, JUNG T P, CHEN Y F, et al. Research Advances in Non-invasive Brain-Computer Interface Control Strategies. Journal of Biomedical Engineering, 2022, 39(5): 1033-1040.)
[16] 孟明,罗志增.基于眼动辅助脑电信号的手部动作分类方法.模式识别与人工智能, 2012, 25(6): 1007-1012.
(MENG M, LUO Z Z.Hand Motion Classification Based on Eye-Moving Assisted EEG. Pattern Recognition and Artificial Intelligence, 2012, 25(6): 1007-1012.)
[17] LI D Y, WEI C, LI S Y, et al. Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion // Proc of the 38th International Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2024: 102822-102864.
[18] JIA M L, TANG L M, CHEN B C, et al. Visual Prompt Tuning // Proc of the 17th European Conference on Computer Vision. Berlin, Germany: Springer, 2022: 709-727.
[19] NICKEL M, KIELA D. Poincaré Embeddings for Learning Hierarchical Representations // Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2017: 6341-6350.
[20] DESAI K, NICKEL M, RAJPUROHIT T, et al. Hyperbolic Image-Text Representations // Proc of the 40th International Conference on Machine Learning. San Diego, USA: JMLR: 7694-7731.
[21] JO S, JEONG W, HEO D W, et al. HyFI: Hyperbolic Feature Interpolation for Brain-Vision Alignment. Proceedings of the AAAI Conference on Artificial Intelligence, 2026, 40(7): 5575-5583.
[22] WANG J Y, ZHANG L, LIN H P, et al. NeuroCLIP: Brain-Inspired Prompt Tuning for EEG-to-Image Multimodal Contrastive Learning[C/OL].[2026-04-16]. https://arxiv.org/pdf/2511.09250.
[23] CHEN Z X, LIU S C, LU R H, et al. MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding[C/OL].[2026-04-16]. https://arxiv.org/pdf/2605.24523.
[24] ZHANG K F, HE L H, KE J J, et al. CognitionCapturerPro: Towards High-Fidelity Visual Decoding from EEG/MEG via Multi-modal Information and Asymmetric Alignment[C/OL].[2026-04-16]. https://arxiv.org/pdf/2603.12722.