Dynamic Parameter Setting Method for Domain Adaptation
ZHANG Yuhong1,2, YU Daoyuan1,2, HU Xuegang1,2
1. School of Computer Science and Information Engineering, Hefei University of Technology, Hefei 230601 2. Key Laboratory of Knowledge Engineering with Big Data, Mi-nistry of Education, Hefei University of Technology, Hefei 230601
Abstract The performance of domain adaptation methods for different tasks is unstable due to its static weight settings for multiple measures during feature shift process. Therefore, a dynamic parameter setting method for domain adaption is proposed. Reproducing Kernel Hilbert space is introduced to learn the invariant space by minimizing the distance between both domains according to the discriminative joint probability distribution. In this process, A-distance is employed to measure the discrepancy ratio of the same labels to the different labels, and this ratio is utilized to adjust the proportion of transferability and discriminability distributions dynamically. With this dynamic parameter settings, better performance is obtained. Experimental results on three image classification datasets show the effectiveness of the proposed method.
Fund:National Natural Science Foundation of China(No.61976077,62076087,62076085)
Corresponding Authors:
ZHANG Yuhong, Ph.D., associate professor. Her research interests include transfer learning and machine learning.
About author:: YU Daoyuan, master student. His research interests include domain adaptation and machine learning. HU Xuegang, Ph.D., professor. His research interests include data mining, Web mining and machine learning.
[1] TAN C Q, SUN F C, KONG T, et al. A Survey on Deep Transfer Learning // Proc of the 27th International Conference on Artificial Networks. Berlin, Germany: Springer, 2018: 270-279. [2] PAN S J, YANG Q. A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359. [3] 庄福振,罗 平,何 清,等.迁移学习研究进展.软件学报, 2015, 26(1): 26-39. (ZHUANG F Z, LUO P, HE Q, et al. Survey on Transfer Learning Research. Journal of Software, 2015, 26(1): 26-39). [4] TZENG E, HOFFMAN J, SAENKO K, et al. Adversarial Discriminative Domain Adaptation // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 2962-2971. [5] LONG M S, WANG J M, DING G G, et al. Adaptation Regularization: A General Framework for Transfer Learning. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(5): 1076-1089. [6] SUN B C, FENG J S, SAENKO K. Return of Frustratingly Easy Domain Adaptation // Proc of the 30th AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2016: 2058-2065. [7] CHEN Y M, SONG S J, LI S, et al. Domain Space Transfer Extreme Learning Machine for Domain Adaptation. IEEE Transactions on Cybernetics, 2019, 49(5): 1909-1922. [8] GONG B Q, SHI Y, SHA F, et al. Geodesic Flow Kernel for Unsupervised Domain Adaptation // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2012: 2066-2073. [9] GANIN Y, LEMPITSKY V. Unsupervised Domain Adaptation by Backpropagation // Proc of the 32nd International Conference on Machine Learning. New York, USA: ACM, 2015: 1180-1189. [10] CLINCHANT S, CSURKA G, CHIDLOVSKII B. A Domain Adaptation Regularization for Denoising Autoencoders // Proc of the 54th Annual Meeting of the Association for Computational Linguistics(Short Papers). Stroudsburg, USA: ACL, 2016, II: 26-31. [11] LONG M S, CAO Z J, WANG J M, et al. Conditional Adversarial Domain Adaptation // Proc of the 32nd Annual Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2018: 1647-1657. [12] ZHU Y, HU X G, ZHANG Y H, et al. Transfer Learning with Stacked Reconstruction Independent Component Analysis. Know-ledge-Based Systems, 2018, 152: 100-106. [13] SHEN J, QU Y R, ZHANG W N, et al. Wasserstein Distance Guided Representation Learning for Domain Adaptation[C/OL]. [2021-04-12]. https://export.arxiv.org/pdf/1707.01217. [14] LONG M S, CAO Y, CAO Z J, et al. Transferable Representation Learning with Deep Adaptation Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(12): 3071-3085. [15] DENG H L, ZHANG L, SHU X. Feature Memory-Based Deep Recurrent Neural Network for Language Modeling. Applied Soft Computing, 2018, 68: 432-446. [16] PAN S J, TSANG I W, KWOK J T, et al. Domain Adaptation via Transfer Component Analysis. IEEE Transactions on Neural Networks, 2011, 22(2): 199-210. [17] WANG J D, FENG W J, CHEN Y Q, et al. Visual Domain Adaptation with Manifold Embedded Distribution Alignment // Proc of the 26th ACM International Conference on Multimedia. New York, USA: ACM, 2018: 402-410. [18] LONG M S, ZHU H, WANG J M, et al. Deep Transfer Learning with Joint Adaptation Networks // Proc of the 34th International Conference on Machine Learning. New York, USA: ACM, 2017: 2208-2217. [19] LONG M S, WANG J M, DING G G, et al. Transfer Feature Learning with Joint Distribution Adaptation // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2013: 2200-2207. [20] ZHANG W, WU D R. Discriminative Joint Probability Maximum Mean Discrepancy(DJP-MMD) for Domain Adaptation [C/OL]. [2021-04-12]. https://arxiv.org/pdf/1912.00320.pdf. [21] 应文豪,王士同,邓赵红,等.基于类分布的领域自适应支持向量机.自动化学报, 2013, 39(8): 1273-1288. (YING W H, WANG S T, DENG Z H, et al. Support Vector Machine for Domain Adaptation Based on Class Distribution. Acta Automatica Sinica, 2013, 39(8): 1273-1288.) [22] WANG J D, CHEN Y Q, HAO S J, et al. Balanced Distribution Adaptation for Transfer Learning // Proc of the IEEE International Conference on Data Mining. Washington, USA: IEEE, 2017: 1129-1134. [23] KURMI V K, KUMAR S, NAMBOODIRI V P. Attending to Discriminative Certainty for Domain Adaptation // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 491-500. [24] GRIFFIN G, HOLUB A, PERONA P. Caltech-256 Object Category Dataset[DB/OL]. [2021-04-12]. https://authors.library.caltech.edu/7694/1/CNS-TR-2007-001.pdf. [25] GROSS R, MATTHEWS I, COHN J F, et al. Multi-PIE // Proc of the 8th IEEE International Conference on Automatic Face and Gesture Recognition. Washington, USA: IEEE, 2008. DOI: 10.1109/AFGR.2008.4813399. [26] XU R J, LIU P L, ZHANG Y, et al. Joint Partial Optimal Transport for Open Set Domain Adaptation // Proc of the 29th International Joint Conference on Artificial Intelligence. Cambridge, USA: Morgan Kaufmann, 2020: 2540-2546.
2021, Vol. 34 
