摘要 谱聚类因其在建模数据间成对相似关系方面的优越性而广泛应用于无监督学习领域.然而,传统谱聚类方法通常依赖干净、结构一致的数据分布,在现实应用中面临常见的噪声样本时,性能显著下降.针对该问题,文中提出融合CLIP(Contrastive Language Image Pretraining)先验知识的谱聚类框架——基于知识重用的噪声环境谱聚类(Noise Spectral Clustering with Assistance of Knowledge Reuse, NSCR).该方法充分利用多模态神经网络在跨模态语义理解上的先验能力,构建基于知识重用的伪标签生成机制,通过多模型语义一致性判别机制与基于信息熵的不确定性建模机制识别高可信样本.同时引入归一化指数熵作为伪标签不确定性度量指标,从多模型输出中筛选语义一致、信息熵较低的样本,并生成伪标签,监督信号形式,引导聚类过程.此外,引入联合优化目标,扩展传统谱聚类方法,通过特征对齐与正则化平衡因子缓解伪标签监督与聚类目标之间的语义冲突.在多个公开数据集上的实验表明,NSCR在不同类型噪声干扰下的鲁棒性与泛化性良好.
Abstract：Spectral clustering is widely used in the field of unsupervised learning due to its superiority in modeling pairwise similarity structures within data. However, conventional spectral clustering methods typically rely on the assumption of clean and structurally consistent datasets, and as a result their performance degrades significantly when the samples are noisy or mismatched in real-world applications. To address these issues, an extended spectral clustering framework, noise spectral clustering based on knowledge reuse(NSCR), is proposed. NSCR effectively leverages semantic knowledge from large-scale multi-model neural networks. A pseudo-label generation module is then designed by leveraging the prior capability of multi-modal neural network models in cross-modal understanding. Reliable samples are identified by semantic-consistency verification and saliency-aware confidence modeling. Softmax entropy is introduced as an uncertainty measure to filter pseudo-labels. Semantically consistent samples with low entropy across multiple models are selected and their pseudo-labels are generated to guide the clustering. Moreover, a joint optimization objective is employed to extend the traditional spectral clustering methods. A feature alignment term and a regularization balancing factor are utilized to mitigate semantic conflicts between pseudo-label supervision and clustering objective. Experiments on public datasets demonstrate that NSCR exhibits good robustness and generalization capability.
[1] DING L, LI C, JIN D, et al. Survey of Spectral Clustering Based on Graph Theory. Pattern Recognition, 2024, 151. DOI: 10.1016/j.patcog.2024.110366. [2] ZHAO Y, LI X L.Deep Spectral Clustering with Regularized Linear Embedding for Hyperspectral Image Clustering. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61. DOI: 10.1109/TGRS.2023.3267070. [3] JIANG R, HAN S S, YU Y M, et al. An Access Control Model for Medical Big Data Based on Clustering and Risk. Information Sciences, 2023, 621: 691-707. [4] FOROUZANDEH S, BERAHMAND K, SHEIKHPOUR R, et al. A New Method for Recommendation Based on Embedding Spectral Clustering in Heterogeneous Networks(RESCHet). Expert Systems with Applications, 2023, 231. DOI: 10.1016/j.eswa.2023.120699. [5] ZHENG Q H.Flexible and Parameter-Free Graph Learning for Multi-view Spectral Clustering. IEEE Transactions on Circuits and Systems for Video Technology, 2024, 34(9): 8966-8971. [6] XIE J, KONG W Y, XIA S Y, et al. An Efficient Spectral Clus-tering Algorithm Based on Granular-Ball. IEEE Transactions on Knowledge and Data Engineering, 2023, 35(9): 9743-9753. [7] MAZARBHUIYA F A, SHENIFY M.A Mixed Clustering Approach for Real-Time Anomaly Detection. Applied Sciences, 2023, 13(7). DOI: 10.3390/app13074151. [8] YUE G L, DENG A S, QU Y P, et al. Stratified Multi-density Spectral Clustering Using Gaussian Mixture Model. Information Sciences, 2023, 633: 182-203. [9] RADFORD A, KIM J W, HALLACY C, et al. Learning Transfe-rable Visual Models from Natural Language Supervision// Proc of the 38th International Conference on Machine Learning. San Diego, USA: JMLR, 2021: 8748-8763. [10] PEI R J, LIU J Z, LI W M, et al. CLIPPING: Distilling CLIP-Based Models with a Student Base for Video-Language Retrieval// Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2023: 18983-18992. [11] GAO Y T, LIU J F, XU Z H, et al. SoftCLIP: Softer Cross-Modal Alignment Makes CLIP Stronger. Proceedings of the AAAI Con-ference on Artificial Intelligence, 2024, 38(3): 1860-1868. [12] LI X, WEN C C, HU Y, et al. RS-CLIP: Zero Shot Remote Sen-sing Scene Classification via Contrastive Vision-Language Supervision. International Journal of Applied Earth Observation and Geoinformation, 2023, 124. DOI: 10.1016/j.jag.2023.103497. [13] YANG H, PAN L Y, YANG Y, et al. LDP: Language-Driven Dual-Pixel Image Defocus Deblurring Network// Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2024: 24078-24087. [14] MACGREGOR P. Fast and Simple Spectral Clustering in Theory and Practice// Proc of the 37th International Conference on Neural Information Processing Systems. Cambridge, USA: MIP Press, 2023: 34410-34425. [15] LIANG H, GUAN H T, ABHADIOMHEN S E, et al. Robust Spectral Clustering via Low-Rank Sample Representation. Applied Computational Intelligence and Soft Computing, 2022. DOI: 10.1155/2022/7540956. [16] BAI L, ZHAO Y X, LIANG J Y.Self-Supervised Spectral Clus-tering with Exemplar Constraints. Pattern Recognition, 2022, 132. DOI: 10.1016/j.patcog.2022.108975. [17] MO Y J, LU Z H, YU R P, et al. Revisiting Self-Supervised Hete-rogeneous Graph Learning from Spectral Clustering Perspective// Proc of the 38th International Conference on Neural Information Processing Systems. Cambridge, USA: MIP Press, 2024: 43133-43163. [18] CAI H M, WANG Y, QI F, et al. Multiview Tensor Spectral Clustering via Co-regularization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(10): 6795-6808. [19] YU Q G, JIA L Q, SHAO Y X, et al. A Local Adaptive Fuzzy Spectral Clustering Method for Robust and Practical Clustering. Scientific Reports, 2025, 15(1). DOI: 10.1038/s41598-025-91812-4. [20] ZHANG J Y, HUANG J X, JIN S, et al. Vision-Language Models for Vision Tasks: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(8): 5625-5644. [21] ZHOU Z Q, LEI Y J, ZHANG B W, et al. ZegCLIP: Towards Adapting CLIP for Zero-Shot Semantic Segmentation// Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2023: 11175-11185. [22] PHAM H, DAI Z H, GHIASI G, et al. Combined Scaling for Zero-Shot Transfer Learning. Neurocomputing, 2023, 555. DOI: 10.1016/j.neucom.2023.126658. [23] MA J W, HUANG P Y, XIE S N, et al. MoDE: CLIP Data Experts via Clustering// Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2024: 26354-26363. [24] LAI Z F, VESDAPUNT N, ZHOU N, et al. PADCLIP: Pseudo-Labeling with Adaptive Debiasing in CLIP for Unsupervised Domain Adaptation// Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2024: 16109-16119. [25] ABDELFATTAH R, GUO Q, LI X G, et al. CDUL: CLIP-Driven Unsupervised Learning for Multi-label Image Classification// Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2023: 1348-1357. [26] CAI S T, QIU L P, CHEN X J,et al. Semantic-Enhanced Image Clustering. Proceedings of the AAAI Conference on Artificial Inte-lligence, 2023, 37(6): 6869-6878. [27] SACHDEVA R, CORDEIRO F R, BELAGIANNIS V, et al, ScanMix: Learning from Severe Label Noise via Semantic Clustering and Semi-supervised Learning. Pattern Recognition, 2023, 134. DOI: 10.1016/j.patcog.2022.109121. [28] ABDULHAMEED T Z, YOUSIF S A, SAMAWI V W, et al. SS-DBSCAN: Semi-supervised Density-Based Spatial Clustering of Applications with Noise for Meaningful Clustering in Diverse Density Data. IEEE Access, 2024, 12: 131507-131520. [29] JIA H J, WU Y H, MAO Q R, et al. Adaptive Density Subgraph Clustering. IEEE Transactions on Computational Social Systems, 2024, 11(4): 5468-5482. [30] LI X Y, FAN H, LIU J L.Noise-Aware Clustering Based on Maximum Correntropy Criterion and Adaptive Graph Regularization. Information Sciences, 2023, 626: 42-59. [31] YANG B, WU J H, ZHANG X T, et al. Fast Correntropy-Based Multi-view Clustering with Prototype Graph Factorization. Information Sciences, 2024, 681. DOI: 10.1016/j.ins.2024.121256. [32] HUANG L T, FAN X Y, XIA T L, et al. SC2-Net: Self-Supervised Learning for Multi-view Complementarity Representation and Consistency Fusion Network. Neurocomputing, 2023, 556. DOI: 10.1016/j.neucom.2023.126695. [33] TREMBLAY N, LOUKAS A.Approximating Spectral Clustering via Sampling: A Review// ROS F, GUILLAUME S, eds. Sampling Techniques for Supervised or Unsupervised Tasks. Berlin, Germany: Springer, 2019: 129-183. [34] KRIZHEVSKY A.Learning Multiple Layers of Features from Tiny Images. Technical Report. Toronto, Canada: University of Toronto, 2009. [35] CHANG J L, WANG L F, MENG G F, et al. Deep Adaptive Image Clustering// Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2017: 5880-5888. [36] XIE J Y, GIRSHICK R, FARHADI A. Unsupervised Deep Embedding for Clustering Analysis[C/OL]. [2025-06-17].https://arxiv.org/abs/1511.06335. [37] CAI J Y, FAN J C, GUO W Z, et al. Efficient Deep Embedded Subspace Clustering// Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 21-30. [38] SADEGHI M, ARMANFARD N. Deep Clustering with Self-Supervision Using Pairwise Similarities[C/OL]. [2025-06-17].https://arxiv.org/abs/2405.03590. [39] WU J L, LONG K Y, WANG F, et al. Deep Comprehensive Co-rrelation Mining for Image Clustering// Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 8149-8158. [40] LI Y F, HU P, LIU Z T, et al. Contrastive Clustering. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 31(10): 8547-8555. [41] HE K M, FAN H Q, WU Y X, et al. Momentum Contrast for Unsupervised Visual Representation Learning// Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 9726-9735. [42] HUANG J B, GONG S G, ZHU X T.Deep Semantic Clustering by Partition Confidence Maximisation// Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 8846-8855. [43] TSAI T W, LI C X, ZHU J. MICE: Mixture of Contrastive Experts for Unsupervised Image Clustering[C/OL]. [2025-06-17]. https://arxiv.org/pdf/2105.01899. [44] TAO Y L, TAKAGI K, NAKATA K. Clustering-Friendly Representation Learning via Instance Discrimination and Feature Decorrelation[C/OL]. [2025-06-17]. https://arxiv.org/abs/2106.00131. [45] DANG Z Y, DENG C, YANG X, et al. Nearest Neighbor Ma-tching for Deep Clustering// Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 13688-13697. [46] NIU C, SHAN H M, WANG G.SPICE: Semantic Pseudo-Labeling for Image Clustering. IEEE Transactions on Image Processing, 2022, 31: 7264-7278. [47] ZHANG H X, LI Y J, HUANG D. Dual-Level Cross-Modal Con-trastive Clustering[C/OL]. [2025-06-17]. https://arxiv.org/abs/2409.04561. [48] LI Y F, HU P, PENG D Z, et al. Image Clustering with External Guidance[C/OL].[2025-06-17]. https://arxiv.org/pdf/2310.11989. [49] LI Z X, JIA Y H, LIU H, et al. Learning from Sample Stability for Deep Clustering[C/OL].[2025-06-17]. https://openreview.net/forum?id=gtVW8CO6Wf. [50] LI M J, CAO W M, YU Z W, et al. Exploring Contrastive Lear-ning and CLIP for Improving Image Clustering. Information Sciences, 2025, 718. DOI: 10.1016/j.ins.2025.122412. [51] CHANG L Y, CHEN L T, ZHOU C.Uncertainty-Aware Contrastive Learning for Deep Clustering. Neurocomputing, 2025, 647. DOI: 10.1016/j.neucom.2025.130568. [52] XU Y K, HUANG D, WANG C D, et al. Deep Image Clustering with Contrastive Learning and Multi-scale Graph Convolutional Networks. Pattern Recognition, 2024, 146. DOI: 10.1016/j.patcog.2023.110065.
2025, Vol. 38 
