1. School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122 2. Jiangsu Key Construction Laboratory of Internet of Things App-lication Technology, Wuxi Taihu University, Wuxi 214064
Abstract Many conventional classification methods start with the hypothesis that the distribution of training samples is same as or at least similar to that of testing samples. In many practical applications, it is difficult to agree with the above hypothesis. And thus the classification performance of some traditional methods, such as support vector machine, is reduced. Therefore, a class-aware based KNN classification method(CA-KNN) is proposed. A sparse representation model is proposed based on the assumption that any testing sample can be represented sparsely by the training samples. The class label information is utilized effectively by CA-KNN to improve the accuracy of the sparse representation. The idea of nearest neighbor classification of KNN is introduced to improve the generalization capability of CA-KNN . And it is proved in theory that CA-KNN classifier is directly related to Bayes decision rule for the minimum error. The experimental and theoretical results show that CA-KNN generates better classification performance.
Fund:National Natural Science Foundation of China(No.61972181,61772198), Natural Science Foundation of Jiangsu Province(No.BK20191331), Internet of Things Application of Jiangsu Province Key Construction Laboratory 2020 Open Project(No.WXWL01)
Corresponding Authors:
WANG Shitong, master, professor. His research interests include artificial intelligence and pattern recognition.
About author:: BIAN Zekang, Ph.D. candidate. His research interests include artificial intelligence and pattern recognition. ZHANG Jin, Ph.D. candidate. His research interests include artificial intelligence and pattern recognition.
[1] CHEN S H, GAO C H. Linear Priors Mined and Integrated for Transparency of Blast Furnace Black-Box SVM Model. IEEE Transactions on Industrial Informatics, 2020, 16(6): 3862-3870. [2] LU Y, CHEUNG Y M, TANG Y Y. Bayes Imbalance Impact Index: A Measure of Class Imbalanced Data Set for Classification Problem. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(9): 3525-3539. [3] LIU H Y, ZHU T W, SHANG F H, et al. Deep Fuzzy Graph Con-volutional Networks for PolSAR Imagery Pixelwise Classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14(1): 504-514. [4] ES-SABERY F, HAIR A, QADIR J, et al. Sentence-Level Classification Using Parallel Fuzzy Deep Learning Classifier. IEEE Access, 2021, 9(1): 17943-17985. [5] FENG S, CHEN C L P, ZHANG C Y. A Fuzzy Deep Model Based on Fuzzy Restricted Boltzmann Machines for High-Dimensional Data Classification. IEEE Transactions on Fuzzy Systems, 2020, 28(7): 1344-1355. [6] ZHANG M L, ZHOU Z H. ML-KNN: A Lazy Learning Approach to Multi-label Learning. Pattern Recognition, 2007, 40(7): 2038-2048. [7] ZOU H, HASTIE T. Regularization and Variable Selection via the Elastic Net. Journal of the Royal Statistical Society(Statistical Metho-dology), 2005, 67(2): 301-320. [8] GONG F K, ZHANG S S, HE Q. Sparse Bayesian Learning for Distributed Passive Radar Imaging via Covariance Sparse Representation // Proc of the 6th Asia-Pacific Conference on Synthetic Aperture Radar. Berlin, Germany: Springer, 2019. DOI: 10.1109/APSAR46974.2019.9048544. [9] LI C L, SUN X, WANG X, et al. Grayscale-Thermal Object Tracking via Multitask Laplacian Sparse Representation. IEEE Transactions on Systems, Man, and Cybernetics(Systems), 2017, 47(4): 673-681. [10] DAI B W, LI H H, WEI L. Image Processing Unit for General-Purpose Representation and Association System for Recognizing Low-Resolution Digits with Visual Information Variability. IEEE Transactions on Systems, Man, and Cybernetics(Systems), 2018, 48(3): 317-328. [11] WANG S S, TAN S, GAO Y, et al. Learning Joint-Sparse Codes for Calibration-Free Parallel MR Imaging. IEEE Transactions on Medical Imaging, 2018, 37(1): 251-261. [12] LIU Q F, LIU C J. A Novel Locally Linear KNN Method with Applications to Visual Recognition. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(9): 2010-2021. [13] SHAFIEE M J, HAIDER S A, WONG A, et al. Apparent Ultra-High b-Value Diffusion-Weighted Image Reconstruction via Hidden Conditional Random Fields. IEEE Transactions on Medical Imaging, 2015, 34(5): 1111-1124. [14] TIBSHIRANI R, SAUNDERS M, ROSSET S, et al. Sparsity and Smoothness via the Fused Lasso. Journal of the Royal Statistical Society(Statistical Methodology), 2005, 67(1): 91-108. [15] WRIGHT J, YANG A Y, GANESH A, et al. Robust Face Recognition via Sparse Representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(2): 210-227. [16] WANG J, ZHANG H Q, WANG J Z, et al. Feature Selection Using a Neural Network with Group Lasso Regularization and Controlled Redundancy. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(3): 1110-1123. [17] ZHANG S R, ZHU Z B, ZHANG B X, et al. Fused Group Lasso: A New EEG Classification Model with Spatial Smooth Constraint for Motor Imagery-Based Brain-Computer Interface. IEEE Sensors Journal, 2021, 21(2): 1764-1778. [18] SUN H, LI S Y, ZHENG X T, et al. Remote Sensing Scene Classification by Gated Bidirectional Network. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(1): 82-96. [19] FRANCE S, CARROLL D. Is the Distance Compression Effect Overstated? Some Theory and Experimentation // Proc of the International Workshop on Machine Learning and Data Mining. Berlin, Germany: Springer, 2009: 280-294. [20] BIAN Z K, ISHIBUCHI H, WANG S T. Joint Learning of Spectral Clustering Structure and Fuzzy Similarity Matrix of Data. IEEE Transactions on Fuzzy Systems, 2019, 27(1): 31-44. [21] ZHANG T, ZHENG W M, CUI Z, et al. Spatial-Temporal Recurrent Neural Network for Emotion Recognition. IEEE Transactions on Cybernetics, 2019, 49(3): 839-847. [22] LEE S J, LEE S, CHO S I, et al. Object Detection-Based Video Retargeting with Spatial-Temporal Consistency. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(12): 4434-4439. [23] YANG M, ZHANG L, FENG X C, et al. Sparse Representation Based Fisher Discrimination Dictionary Learning for Image Classification. International Journal of Computer Vision, 2014, 109(3): 209-232. [24] BI P F, DU X. Application of Locally Invariant Robust PCA for Underwater Image Recognition. IEEE Access, 2021, 9(1): 29470-29481. [25] CHEN X, LUO X Z, WENG J, et al. Multi-view Gait Image Generation for Cross-View Gait Recognition. IEEE Transactions on Image Processing, 2021, 30: 3041-3055. [26] XU Y W, GAO X T, LIN S, et al. Automatic Grading of Nuclear Cataracts from Slit-Lamp Lens Images Using Group Sparsity Regression // Proc of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2013: 468-475. [27] CHENG J, YIN F S, WONG D W K, et al. Sparse Dissimilarity-Constrained Coding for Glaucoma Screening. IEEE Transactions on Biomedical Engineering, 2015, 62(5): 1395-1403. [28] WANG J J, YANG J C, YU K, et al. Locality-Constrained Linear Coding for Image Classification // Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2010: 3360-3367. [29] YANG J C, YU K, GONG Y H, et al. Linear Spatial Pyramid Matching Using Sparse Coding for Image Classification // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2009: 1974-1801. [30] TIBSHIRANI R. Regression Shrinkage and Selection via the Lasso: A Retrospective. Journal of the Royal Statistical Society (Statistical Methodology), 2011, 73(3): 273-282. [31] ZHONG L W, KWOK J T. Efficient Sparse Modeling with Automatic Feature Grouping. IEEE Transactions on Neural Networks and Learning Systems, 2012, 23(9): 1436-1447. [32] WRIGHT J, MA Y, MAIRAL J, et al. Sparse Representation for Computer Vision and Pattern Recognition. Proceedings of the IEEE, 2010, 98(6): 1031-1044. [33] YANG W H, LIU J Y, YANG S, et al. Image Super-Resolution via Nonlocal Similarity and Group Structured Sparse Representation // Proc of the IEEE Conference on Visual Communications and Image Processing. Washington, USA: IEEE, 2015. DOI: 10.1109/VCIP.2015.7457822. [34] ARGYRIOU A, EVGENIOU T, PONTIL M. Convex Multi-task Fea-ture Learning. Machine Learning, 2008, 73(3): 243-272. [35] SHAN H M, ZHANG J P, KRUGER U. Learning Linear Representation of Space Partitioning Trees Based on Unsupervised Kernel Dimension Reduction. IEEE Transactions on Cybernetics, 2016, 46(12): 3427-3438. [36] DUMONT C, KLUYSKENS V, DEHEZ B. Linear State-Space Re-presentation of Heteropolar Electrodynamic Bearings with Radial Magnetic Field. IEEE Transactions on Magnetics, 2016, 52(1): 1-9. [37] MARONNA R A, MARTIN D R, YOHAI V J. Robust Statistics: Theory and Methods. New York, USA: John Wiley & Son, 2006. [38] SCOTT D W. Multivariate Density Estimation: Theory, Practice, and Visualization. New York, USA: John Wiley & Son, 2008. [39] FAWCETT T. An Introduction to ROC Analysis. Pattern Recognition Letters, 2006, 27(8): 861-874. [40] DEMSAR J. Statistical Comparisons of Classifiers over Multiple Data Sets. Journal of Machine Learning Research, 2006, 7: 1-30.
2021, Vol. 34 
