Abstract Neighborhood entropy is adopted as the sample selection criteria in active learning. The example with the highest entropy value is considered as the most uncertain one based on current nearest neighbor rule. And labeling the most uncertain example can achieve higher accuracy with fewer samples. An active learning algorithm based on neighborhood entropy is proposed. The scheme estimates entropy value of neighbor unlabeled sample and label the sample with the highest value. Experimental results show the example selection based on neighborhood entropy achieves higher accuracy compared with maximal distance sampling and random sampling.
[1] Angluin D. Queries and Concept Learning. Machine Learning, 1988, 2(4): 319-342 [2] Lewis D D, Catlett J. Heterogeneous Uncertainty Sampling for Supervised Learning // Proc of the 11th International Conference on Machine Learning. New Brunswick, USA, 1994: 148-156 [3] Freund Y, Seung H S, Shamir E, et al. Selective Sampling Using the Query by Committee Algorithm. Machine Learning, 1997, 28(2/3): 133-168 [4] Seung H S, Opper M, Sompolinsky H. Query by Committee // Proc of the 5th Annual Workshop on Computational Learning Theory. Pittsburgh, USA, 1992: 287-294 [5] Schohn G, Cohn D. Less Is More: Active Learning with Support Vector Machines // Proc of the 17th International Conference on Machine Learning. San Francisco, USA, 2000: 839-846 [6] Roy N. Mccallum A. Toward Optimal Active Learning through Sampling Estimation of Error // Proc of the 18th International Conference on Machine Learning. San Francisco, USA, 2001: 441-448 [7] Cohn D, Atlas L, Ladner R. Improving Generalization with Active Learning. Machine Learning: Special Issue on Structured Connectionist Systems, 1994, 15(2): 201-221 [8] Lindenbaum M, Markovitch S, Rusakov D. Selective Sampling for Nearest Neighbor Classifiers. Machine Learning, 2004, 54(2): 125-152 [9] Valiant L G. A Theory of the Learnable. Communications of the ACM, 1984, 27(11): 1134-1142 [10] Cover T, Hart P. Nearest Neighbor Pattern Classification. IEEE Trans on Information Theory, 1967, 13(1): 21-27 [11] Hart P E. The Condensed Nearest Neighbor Rule. IEEE Trans on Information Theory, 1968, 14(3): 515-516 [12] Hu Qinghua, Yu Daren, Xie Zongxin. Neighborhood Classifiers. Expert Systems with Applications, 2008, 34(2): 866-876 [13] Wang Hui. Nearest Neighbors by Neighborhood Counting. IEEE Trans on Pattern Analysis and Machine Intelligence, 2006, 28(6): 942-953 [14] Shannon C E. A Mathematical Theory of Communication. Bell System Technical, 1948, 27: 379-433, 623-656 [15] Hasenjager M, Ritter H. Active Learning with Local Models. Neural Processing Letters, 1998, 7(2): 107-117
2011, Vol. 24 
