Abstract Hyperspectral image(HSI) data used for training in deep learning are insufficient, and therefore deeper network is unfavorable for spectral-spatial feature extraction. To solve this problem, a 3D multi-scale feature fusion residual network is proposed. Spectral-spatial features are learned by deep learning and multi-scale feature fusion. Firstly, the dimension of 3D-HSI data is adaptively reduced, and the images after dimensionality reduction are used as the input of the network. Secondly, spectral-spatial features are extracted successively through multi-scale feature fusion residual blocks and features of different scales are fused. The information flow is enhanced through sharing features and richer features are obtained. Finally, the network is trained end-to-end and tested on corresponding datasets. Experimental results show the satisfactory classification performance of the proposed network.
[1] LANDGREBE D. Hyperspectral Image Data Analysis. IEEE Signal Processing Magazine, 2002, 19(1): 17-28. [2] BIOUCAS-DIAS J M, PLZAZ A, CAMPS-VALLS G, et al. Hyperspectral Remote Sensing Data Analysis and Future Challenges. IEEE Geoscience and Remote Sensing Magazine, 2013, 1(2): 6-36. [3] ZHANG X, SUN Y L, SHANG K, et al. Crop Classification Based on Feature Band Set Construction and Object-Oriented Approach Using Hyperspectral Images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2016, 9(9): 4117-4128. [4] UZKENT B, RANGNEKAR A, HOFFMAN M J. Aerial Vehicle Tracking by Adaptive Fusion of Hyperspectral Likelihood Maps // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 1998: 233-242. [5] CARTER G A, LUCAS K L, BLOSSOM G A, et al. Remote Sen-sing and Mapping of Tamarisk Along the Colorado River, USA: A Comparative Use of Summer-Acquired Hyperion, Thematic Mapper and Quickbird Data. Remote Sensing, 2009, 1(3): 318-329. [6] ATIYA A. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. IEEE Transactions on Neural Networks, 2005, 16(3). DOI: 10.1109/TNN.2005.848998. [7] 曹 扬,赵慧洁,黄四牛,等.基于高效置信传播的改进马尔可夫随机场高光谱数据分类算法.模式识别与人工智能, 2014, 27(3): 248-255. (CAO Y, ZHAO H J, HUANG S N, et al. An Improved Markov Random Field Classification Approach for Hyperspectral Data Based on Efficient Belief Propagation. Pattern Recognition and Artificial Intelligence, 2014, 27(3): 248-255.) [8] 孙立新,高 文.基于粗糙集的遥感优化分类波段选择.模式识别与人工智能, 2000, 13(2): 181-186. (SUN L X, GAO W. Selecting the Optimal Classification Bands Based on Rough Sets. Pattern Recognition and Artificial Intelligence, 2000, 13(2): 181-186.) [9] CHEN Y, NASRABADI N M, TRAN T D. Hyperspectral Image Classification Using Dictionary-Based Sparse Representation. IEEE Transactions on Geoscience and Remote Sensing, 2011, 49(10): 3973-3985. [10] LIU J J, WU Z B, WEI Z H, et al. Spatial-Spectral Kernel Sparse Representation for Hyperspectral Image Classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2013, 6(6): 2462-2471. [11] YUAN H L. Robust Patch-Based Sparse Representation for Hyperspectral Image Classification. International Journal of Wavelets Multiresolution and Information Processing, 2017, 15(3): 1351-1363. [12] HERN／ANDEZ-ESPINOSA C, FERNÁNDEZ-REDONDO M, TORRES- SOSPEDRA J. Some Experiments with Ensembles of Neural Networks for Classification of Hyperspectral Images // Proc of the International Symposium on Neural Networks. Berlin, Germany: Springer, 2004: 912-917. [13] SLAVKOVIKJ V, VERSTOCKT S, DE NEVE W, et al. Hyperspectral Image Classification with Convolutional Neural Networks // Proc of the 23rd ACM International Conference on Multimedia. New York, USA: ACM, 2015: 1159-1162. [14] HU W, HUANG Y Y, WEI L, et al. Deep Convolutional Neural Networks for Hyperspectral Image Classification. Journal of Sensors, 2015. DOI: 10.1155/2015/258619. [15] ZHU J, FANG L Y, GHAMISI P. Deformable Convolutional Neural Networks for Hyperspectral Image Classification. IEEE Geoscience and Remote Sensing Letters, 2018, 15(8): 1254-1258. [16] YANG X F, YE Y M, LI X T, et al. Hyperspectral Image Classification with Deep Learning Models. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(9): 5408-5423. [17] XU Y H, ZHANG L P, DU B, et al. Spectral-Spatial Unified Networks for Hyperspectral Image Classification. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(10): 5893-5909. [18] HOCHREITER S, SCHMIDHUBER J. Long Short-Term Memory. Neural Computation, 1997, 9(8): 1735-1780. [19] ZHONG Z, LI J, LUO Z M, et al. Spectral-Spatial Residual Network for Hyperspectral Image Classification: A 3-D Deep Learning Framework. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(2): 847-858. [20] LUO Y N, ZOU J, YAO C F, et al. HSI-CNN: A Novel Convolution Neural Network for Hyperspectral Image // Proc of the International Conference on Audio Language and Image Processing. Berlin, Germany: Springer, 2018: 464-469. [21] SHAMSOLMOALI P, ZAREAPOOR M, YANG J. Convolutional Neural Network in Network(CNNiN): Hyperspectral Image Classification and Dimensionality Reduction. IET Image Processing, 2019, 13(2): 246-253. [22] HE M Y, LI B, CHEN H H. Multi-scale 3D Deep Convolutional Neural Network for Hyperspectral Image Classification // Proc of the IEEE International Conference on Image Processing. Washington, USA: IEEE, 2017: 3904-3908. [23] LI J C, FANG F M, MEI K F, et al. Multi-scale Residual Network for Image Super-Resolution // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 527-542. [24] GONG Z Q,ZHONG P,YU Y,et al. A CNN withMulti-scale Convolution and Diversified Metric for Hyperspectral Image Classification. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(6): 3599-3618. [25] XIONG Z T, YUAN Y, WANG Z. AI-NET: Attention Inception Neural Networks for Hyperspectral Image Classification // Proc of the IEEE International Geoscience and Remote Sensing Symposium. Washington, USA: IEEE, 2018: 2647-2650. [26] LI Y, ZHANG H K, SHEN Q. Spectral-Spatial Classification of Hyperspectral Imagery with 3D Convolutional Neural Network. Remote Sensing, 2017, 9(1): 67-88. [27] KUO B C, HO H H, LI C H, et al. A Kernel-Based Feature Selection Method for SVM with RBF Kernel for Hyperspectral Image Classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(1): 317-326. [28] LUO H W, TANG Y Y, WANG Y L, et al. Hyperspectral Image Classification Based on Spectral Spatial One-Dimensional Manifold Embedding. IEEE Transactions on Geoscience and Remote Sen-sing, 2016, 54(9): 5319-5340. [29] WANG Q, GAO J Y, YUAN Y. A Joint Convolutional Neural Networks and Context Transfer for Street Scenes Labeling. IEEE Transactions on Intelligent Transportation Systems, 2018, 19(5): 1457-1470.
2019, Vol. 32 
