Abstract The existing image classification methods depend on large-scale manually annotated data. However, when data is limited, these methods suffer from deficiencies in both local feature representation and the number of samples. To address these issues, a method for few-shot image classification based on local contrastive learning and novel class feature generation is proposed. First, local contrastive learning is introduced to represent images as multiple local features and conduct supervised contrastive learning among these local features. Thus, the model capability to represent local features is enhanced. Second, global contrastive learning is employed to ensure the separability of the overall image features. Finally, a feature generation method is proposed to mitigate the data scarcity issue under few-shot conditions. Experiments on public datasets demonstrate the superiority of the proposed method.
Fund:National Natural Science Foundation of China(No.62372155 ), Aeronautical Science Foundation of China(No.2022Z071108001), Joint Fund of Ministry of Education for Equipment Pre-research(No.8091B022123), Qinglan Project of Jiangsu Province
Corresponding Authors:
LIU Fan, Ph.D., professor. His research interests include computer vision, multimedia analysis and understanding.
About author:: CHEN Ning, Master student. His research interests include computer vision, image cla-ssification and object detection. DONG Chenwei, Master student. His research interests include deep learning and noisy correspondence. CHEN Zhiyu, Master student. His research interests include computer vision and image classification.
CHEN Ning,LIU Fan,DONG Chenwei等. Few-Shot Image Classification Based on Local Contrastive Learning and Novel Class Feature Generation[J]. Pattern Recognition and Artificial Intelligence, 2024, 37(10): 936-946.
[1] SONG C F, HUANG Y, OUYANG W L, et al. Mask-Guided Con-trastive Attention Model for Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 1179-1188. [2] ANTONELLI S, AVOLA D, CINQUE L, et al. Few-Shot Object Detec-tion: A Survey. ACM Computing Surveys(CSUR), 2022, 54(11s). DOI: 10.1145/351902. [3] WANG K X, LIEW J H, ZOU Y T, et al. PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 9196-9205. [4] VALMADRE J, BERTINETTO L, HENRIQUES J, et al. End-to-End Representation Learning for Correlation Filter Based Tracking // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 5000-5008. [5] PRABHU V, KANNAN A, RAVURI M, et al. Few-Shot Learning for Dermatological Disease Diagnosis // Proc of the Machine Lear-ning for Healthcare Conference. New York, USA: ACM, 2019: 235-252. [6] HE K M, FAN H Q, WU Y X, et al. Momentum Contrast for Unsupervised Visual Representation Learning // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 9726-9735. [7] CHEN T, KORNBLITH S, NOROUZI M, et al. A Simple Framework for Contrastive Learning of Visual Representations // Proc of the 37th International Conference on Machine Learning. New York, USA: ACM, 2020: 1597-1607. [8] YE M, ZHANG X, YUEN P C, et al. Unsupervised Embedding Learning via Invariant and Spreading Instance Feature // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 6203-6212. [9] WU Z R, XIONG Y J, YU S X, et al. Unsupervised Feature Lear-ning via Non-parametric Instance Discrimination // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 3733-3742. [10] TIAN Y L, KRISHNAN D, ISOLA P.Contrastive Multiview Co-ding // Proc of the 16th European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 776-794. [11] CHEN X L, FAN H Q, GIRSHICK R, et al. Improved Baselines with Momentum Contrastive Learning[C/OL].[2024-05-08]. https://arxiv.org/pdf/2003.04297. [12] CHEN T, KORNBLITH S, SWERSKY K, et al. Big Self-Supervised Models Are Strong Semi-Supervised Learners // Proc of the 34th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2020: 22243-22255. [13] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An Image Is Worth 16×16 Words: Transformers for Image Recognition at Scale[C/OL].[2024-05-08]. https://arxiv.org/pdf/2010.11929v2. [14] CHEN X L, XIE S N, HE K M.An Empirical Study of Training Self-Supervised Vision Transformers // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2021: 9620-9629. [15] CARON M, TOUVRON H, MISRA I, et al. Emerging Properties in Self-Supervised Vision Transformers // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2021: 9630-9640. [16] KHOSLA P, TETERWAK P, WANG C, et al. Supervised Con-trastive Learning // Proc of the 34th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2020: 18661-18673. [17] DAS R, WANG Y X, MOURA J M F. On the Importance of Distractors for Few-Shot Classification // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2021: 9010-9020. [18] LIU C, FU Y W, XU C M, et al. Learning a Few-Shot Embedding Model with Contrastive Learning. Proceedings of the AAAI Confe-rence on Artificial Intelligence, 2021, 35(10): 8635-8643. [19] OUALI Y, HUDELOT C, TAMI M.Spatial Contrastive Learning for Few-Shot Classification // Proc of the European Conference on Machine Learning and Knowledge Discovery in Databases. Berlin, Germany: Springer, 2021: 671-686. [20] MA J W, XIE H C, HAN G X, et al. Partner-Assisted Learning for Few-Shot Image Classification // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2021: 10553-10562. [21] CHAWLA N V, BOWYER K W, HALL L O, et al. SMOTE: Synthetic Minority Over-Sampling Technique. Journal of Artificial Intelligence Research, 2002, 16(1): 321-357. [22] WANG T Z, ISOLA P.Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere. Journal of Machine Learning Research, 2020, 119: 9929-9939. [23] SNELL J, SWERSKY K, ZEMEL R.Prototypical Networks for Few-Shot Learning // Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2017: 4080-4090. [24] ZHANG C, CAI Y J, LIN G S, et al. DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(5): 5632-5648. [25] GIDARIS S, BURSUC A, KOMODAKIS N, et al. Boosting Few-Shot Visual Learning with Self-Supervision // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 8058-8067. [26] ZHOU Z Q, QIU X, XIE J T, et al. Binocular Mutual Learning for Improving Few-Shot Classification // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2021: 8382-8391. [27] KANG D, KWON H, MIN J H, et al. Relational Embedding for Few-Shot Classification // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2021: 8802-8813. [28] WERTHEIMER D, TANG L M, HARIHARAN B.Few-Shot Cla-ssification with Feature Map Reconstruction Networks // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 8008-8017. [29] BAIK S, CHOI J, KIM H, et al. Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2021: 9445-9454. [30] XU C M, FU Y W, LIU C, et al. Learning Dynamic Alignment via Meta-Filter for Few-Shot Learning // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 5178-5187. [31] TANG S X, CHEN D P, BAI L, et al. Mutual CRF-GNN for Few-Shot Learning // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 2329-2339. [32] ZHANG M L, ZHANG J H, LU Z W, et al. IEPT: Instance-Level and Episode-Level Pretext Tasks for Few-Shot Learning[C/OL].[2024-05-08]. https://openreview.net/pdf?id=xzqLpqRzxLq. [33] MA R K, FANG P F, DRUMMOND T, et al. Adaptive Poincaré Point to Set Distance for Few-Shot Classification. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(2): 1926-1934. [34] YU T Y, HE S, SONG Y Z, et al. Hybrid Graph Neural Networks for Few-Shot Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(3): 3179-3187. [35] AFRASIYABI A, LAROCHELLE H, LALONDE J F, et al. Ma-tching Feature Sets for Few-Shot Image Classification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 9004-9014. [36] LIU Y, ZHANG W F, XIANG C, et al. Learning to Affiliate: Mutual Centralized Learning for Few-Shot Classification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 14391-14400. [37] LAI J X, YANG S Q, WU W L, et al. SpatialFormer: Semantic and Target Aware Attentions for Few-Shot Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 2023, 37(7): 8430-8437. [38] BAIK S, CHOI M, CHOI J, et al. Learning to Learn Task-Adaptive Hyperparameters for Few-Shot Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(3): 1441-1454. [39] CHENG H, YANG S Y, ZHOU J T, et al. Frequency Guidance Matters in Few-Shot Learning // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2023: 11780-11790. [40] SUNG F, YANG Y X, ZHANG L, et al. Learning to Compare: Relation Network for Few-Shot Learning // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 1199-1208. [41] LI K, ZHANG Y L, LI K P, et al. Adversarial Feature Hallucination Networks for Few-Shot Learning // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 13467-13476. [42] ZHANG C, DING H H, LIN G S, et al. Meta Navigator: Search for a Good Adaptation Policy for Few-Shot Learning // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2021: 9415-9424. [43] YANG Z Y, WANG J H, ZHU Y Y.Few-Shot Classification with Contrastive Learning // Proc of the 18th European Conference on Computer Vision. Berlin, Germany: Springer, 2022: 293-309. [44] LIU B, CAO Y, LIN Y T, et al. Negative Margin Matters: Understanding Margin in Few-Shot Classification // Proc of the 16th European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 438-455. [45] MANGLA P, KUMARI N, SINHA A, et al. Charting the Right Manifold: Manifold Mixup for Few-Shot Learning // Proc of the IEEE Winter Conference on Applications of Computer Vision. Washington, USA: IEEE, 2020: 2207-2216. [46] TIAN Y L, WANG Y, KRISHNAN D, et al. Rethinking Few-Shot Image Classification: A Good Embedding Is All You Need? // Proc of the 16th European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 266-282. [47] RAJASEGARAN J, KHAN S, HAYAT M, et al. Self-Supervised Knowledge Distillation for Few-Shot Learning[C/OL].[2024-05-08]. https://arxiv.org/pdf/2006.09785. [48] RIZVE M N, KHAN S, KHAN F S, et al. Exploring Complementary Strengths of Invariant and Equivariant Representations for Few-Shot Learning // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 10831-10841. [49] WU J M, ZHANG T Z, ZHANG Y D, et al. Task-Aware Part Mining Network for Few-Shot Learning // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2021: 8413-8422. [50] XIE J T, LONG F, LÜ J M, et al. Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 7962-7971. [51] KIM J, KIM H, KIM G.Model-Agnostic Boundary-Adversarial Sampling for Test-Time Generalization in Few-Shot Learning // Proc of the 16th European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 599-617. [52] SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2017: 618-626. [53] VAN DER MAATEN L, HINTON G. Visualizing Data Using t-SNE. Journal of Machine Learning Research, 2008, 9: 2579-2605.
2024, Vol. 37 
