A Hybrid Learning Algorithm for Elliptical Basis Function Neural Networks
XING HongJie1,2,3, WANG Yong1,2, HU BaoGang1,2
1.National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100080 2. Graduate School, Chinese Academy of Sciences, Beijing 1000803. 3.College of Mathematics and Computer Science, Hebei University, Baoding 071002
Abstract A hybrid learning method for the elliptical basis function neural network (EBFNN) is presented. Firstly, the parameters of elliptical basis function (EBF) units in the hidden layer of the EBFNN are initialized by the expectationmaximization (EM) algorithm, while the connection weights plus bias term is initialized by the linear leastsquared method. Then, the gradient descent based optimization procedure adjusts all the parameters simultaneously. The comparison results show that the gradient descent elliptical basis function neural network (GDEBFNN) trained by the proposed hybrid learning method upon the test datasets has higher accuracy than the other three related models. Compared with support vector machine (SVM), the GDEBFNN can achieve comparable generalization ability. Moreover, the GDEBFNN obtains better generalization performance than the decision tree constructed by the Adaboost method.
[1] Haykin S. Neural Networks: A Comprehensive Foundation. 2nd Edition. New York, USA: Prentice Hall, 1999 [2] Lampriello F, Sciandrone M. Efficient Training of RBF Neural Networks for Pattern Recognition. IEEE Trans on Neural Networks, 2001, 12(5): 12351242 [3] Wedding D K, Cios K J. Time Series Forecasting by Combining RBF Networks, Certainty Factors, and the BoxJenkins Model. Neurocomputing, 1996, 10(2): 149168 [4] VakilBaghmisheh M T, Pavesic N. Training RBF Networks with Selective Backpropagation. Neurocomputing, 2004, 62(1): 3964 [5] Karayiannis N B. Reformulated Radial Basis Neural Networks Trained by Gradient Descent. IEEE Trans on Neural Networks, 1999, 10(3): 657671 [6] Xiao Di, Hu Shousong. Ellipsoidal Basis Functional Neural Network Based on Rough KMeans. Journal of Nanjing University of Aeronautics & Astronautics, 2006, 38(3): 321325 (in Chinese) (肖 迪,胡寿松.一种基于粗糙K均值的椭球基函数神经网络. 南京航空航天大学学报, 2006, 38(3): 321325) [7] Mak M W, Kung S Y. Estimation of Elliptical Basis Function Parameters by the EM Algorithm with Application to Speaker Verification. IEEE Trans on Neural Networks, 2000, 11(4): 961969 [8] Zhao Xiang, Zhou Shaoqi, Xiao Deyun. Improved Ellipsoidal Unit Neural Networks and Its Applications in CSTR. Journal of Chongqing University: Natural Science Edition, 2002, 25(5): 5863 (in Chinese) (赵 翔,周绍琦,萧德云.改进的椭球单元网络及其在故障诊断中的应用.重庆大学学报:自然科学版, 2002, 25(5): 5863) [9] Luo Jiancheng, Chen Qiuxian, Zheng Jiang, et al. An Elliptical Basis Function Network for Classification of RemoteSensing Images // Proc of the IEEE International Symposium on Geoscience and Remote Sensing. Toulouse, France, 2003, Ⅵ: 34893494 [10] Yiu K K, Mak M W, Li C K. Gaussian Mixture Models and Probabilistic DecisionBased Neural Networks for Pattern Classification: A Comparative Study. Neural Computing & Applications, 1999, 8(3): 235245 [11] Blake C, Merz C. UCI Repository of Machine Learning Datasets [DB/OL]. [19980101]. http://www.ics.uci.edu/~mlearn/MLRepository.html [12] Vapnik V N. The Nature of Statistical Learning Theory. New York, USA: SpringerVerlag, 1995
2008, Vol. 21 
