Fast Few-Shot Learning Algorithm Based on Deep Network
DAI Leichao1, FENG Lin1, SHANG Xinglin1, SU Han1, GONG Xun2
1. College of Computer Science, Sichuan Normal University, Chengdu 610101 2. School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756
Abstract The cognitive process of the few-shot learning method simulating human learning from a small number of samples is one of the hotspots in the machine learning field. To solve the problems of large task volume and serious overfitting in the iterative process of the current few-shot learning methods, a fast few-shot learning algorithm based on deep network is proposed. Firstly, the kernel density estimation and image filtering methods are utilized to add multiple types of random noise to the training set to generate support sets and query sets. Then, the prototype network is applied to extract the image features of the support set and query set. According to the Bregman divergence, the center point of the support sample of each type of support set is employed as the class prototype. Then, the L2 norm is utilized to measure the distance between the support set and the query image. Multiple heterogeneous base classifiers are generated using cross-entropy feedback loss. Finally, the voting mechanism is introduced to fuse the nonlinear classification results of the base classifiers. Experiments show that the proposed algorithm speeds up the convergence of few-shot learning with higher classification accuracy and strong robustness.
Fund:National Natural Science Foundation of China(No.61876158), Fundamental Research Funds for the Central Universities(No.2682021ZTPY030)
Corresponding Authors:
FENG Lin, Ph.D., professor. His research interests include machine learning and data mining.
About author:: DAI Leichao, master student. His research interests include computer vision and pattern recognition. SHANG Xinglin, master student. Her research interests include machine learning. SU Han, Ph.D., professor. Her research interests include pattern recognition and image processing. GONG Xun, Ph.D., professor. His research interests include computer vision, pa-ttern recognition and medical image proce-ssing.
[1] LI F F, FERGUS R, PERONA P. One-Shot Learning of Object Ca-tegories. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(4): 594-611. [2] FINN C, ABBEEL P, LEVINE S. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks // Proc of the 34th International Conference on Machine Learning. New York, USA: ACM, 2017: 1126-1135. [3] LI Z G, ZHOU F W, CHEN F, et al. Meta-SGD: Learning to Learn Quickly for Few-Shot Learning[C/OL]. [2021-06-15]. https://arxiv.org/pdf/1707.09835.pdf. [4] LIU Y B, LEE J, PARK M, et al. Learning to Propagate Labels: Transductive Propagation Network for Few-Shot Learning[C/OL]. [2021-06-15]. https://openreview.net/pdf?id=SyVuRiC5K7. [5] ZHANG R X, CHE T, GHAHRAMANI Z, et al. MetaGAN: An Adversarial Approach to Few-Shot Learning // Proc of the 32nd International Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2018: 2371- 2380. [6] GREGORY K, RICHARD Z, RUSLAN S. Siamese Neural Networks for One-Shot Image Recognition[C/OL]. [2021-06-15]. http://www.cs.utoronto.ca/~rsalakhu/papers/oneshot1.pdf. [7] VINYALS O, BLUNDELL C, LILLICRAP T, et al. Matching Networks for One Shot Learning // Proc of the 30th International Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2016: 3637-3645. [8] SUNG F, YANG Y X, ZHANG L, et al. Learning to Compare: Relation Network for Few-Shot Learning // Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 1199-1208. [9] CAI Q, PAN Y W, YAO T, et al. Memory Matching Networks for One-Shot Image Recognition // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 4080-4088. [10] 蒋留兵,周小龙,姜风伟,等.基于改进匹配网络的单样本学习.系统工程与电子技术, 2019, 41(6): 1210-1217. (JIANG L B, ZHOU X L, JIANG F W, et al. One-Shot Learning Based on Improved Matching Network. Systems Engineering and Electronics, 2019, 41(6): 1210-1217.) [11] SNELL J, SWERSKY K, ZEMEL R S. Prototypical Networks for Few-Shot Learning // Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2017: 4077-4087. [12] REN M Y, TRIANTAFILLOU E, RAVI S, et al. Meta-Learning for Semi-supervised Few-Shot Classification[C/OL]. [2021-06-15]. https://arxiv.org/pdf/1803.00676.pdf. [13] GARCIA V, BRUNA J. Few-Shot Learning with Graph Neural Network[C/OL]. [2021-06-15]. https://arxiv.org/pdf/1711.04043.pdf. [14] 张鲁宁,左 信,刘建伟.零样本学习研究进展.自动化学报, 2020, 46(1): 1-23. (ZHANG L N, ZUO X, LIU J W. Research and Development on Zero-Shot Learning. Acta Automatica Sinica, 2020, 46(1): 1-23.) [15] ANTOL S, ZITNICK C. L, PARIKH D. Zero-Shot Learning via Visual Abstraction // Proc of the European Conference on Compu-ter Vision. Berlin, Germany: Springer, 2014: 401-416. [16] ROBYNS P, MARIN E, LAMOTTE W, et al. Physical-Layer Fingerprinting of LoRa Devices Using Supervised and Zero-Shot Lear-ning // Proc of the 10th ACM Conference on Security and Privacy in Wireless and Mobile Networks. New York, USA: ACM, 2017: 58-63. [17] YANG Y, LUO Y D, CHEN W L, et al. Zero-Shot Hashing via Transferring Supervised Knowledge // Proc of the 24th ACM International Conference on Multimedia. New York, USA: ACM, 2016: 1286-1295. [18] 林克正,李昊天,白婧轩,等.语义自编码结合关系网络的零样本图像识别算法.模式识别与人工智能, 2019, 32(3): 214-224. (LIN K Z, LI H T, BAI J X, et al. Zero-Shot Image Recognition Algorithm via Semantic Auto-Encoder Combining Relation Network. Pattern Recognition and Artificial Intelligence, 2019, 32(3): 214-224.) [19] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet Cla-ssification with Deep Convolutional Neural Networks. Communications of the ACM, 2017, 60(6): 84-90. [20] BANERJEE A, MERUGU S, DHILLON I S, et al. Clustering with Bregman Divergences. Journal of Machine Learning Research, 2005, 6: 1705-1749. [21] LAKE B, SALAKHATDINOV R, GROSS J, et al. One Shot Lear-ning of Simple Visual Concepts[C/OL]. [2021-06-15]. https://web.mit.edu/jgross/Public/lake_etal_cogsci2011.pdf. [22] KAISER Ł, NACHUM O, ROY A, et al. Learning to Remember Rare Events[C/OL]. [2021-06-15]. https://arxiv.org/pdf/1703.03129v1.pdf.
2021, Vol. 34 
