Abstract To solve the spatial-temporal fusion problem of images of surface reflectivity remote sensing satellites Landsat and MODIS, a spatial-temporal fusion algorithm for remote sensing images based on multi-input dense connected neural network is proposed. Firstly, a multi-input dense connected neural network is put forward to study the remote sensing images containing the difference information between continuous moments. Then, two transition images learned from the network are fused with the two known high spatial resolution images based on the difference similarity hypothesis to obtain the final predicted images. According to the fusion experiment of Landsat remote sensing images and MODIS remote sensing images, the proposed algorithm produces promising results in each quantitative index. The final predicted image by the proposed algorithm is more robust to noise with better recovered detail information.
[1] GAO F, MASEK J, SCHWALLER M, et al. On the Blending of the Landsat and MODIS Surface Reflectance: Predicting Daily Landsat Surface Reflectance. IEEE Transactions on Geoscience and Remote Sensing, 2006, 44(8): 2207-2218. [2] ZHU X L, CHEN J, GAO F, et al. An Enhanced Spatial and Temporal Adaptive Reflectance Fusion Model for Complex Heterogeneous Regions.Remote Sensing of Environment, 2010, 114(11): 2610-2623. [3] HUANG B, SONG H H. Spatiotemporal Reflectance Fusion via Spa-rse Representation.IEEE Transactions on Geoscience and Remote Sensing, 2012, 50(10): 3707-3716. [4] WU B, HUANG B, ZHANG L P. An Error-Bound-Regularized Spa-rse Coding for Spatiotemporal Reflectance Fusion. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(12): 6791-6803. [5] KRIZHEVSKY A, SUTKEVER I, HINTON G E. ImageNet Classification with Deep Convolutional Neural Networks. Communications of the ACM, 2017, 60(6): 84-90. [6] HE K M, ZHANG X Y, REN S Q, et al. Deep Residual Learning for Image Recognition // Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington, USA: IEEE, 2016: 770-778. [7] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2014: 580-587. [8] LONG J, SHELHAMER E, DARRELL T. Fully Convolutional Networks for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651. [9] LIN M, CHEN Q, YAN S C. Network in Network[C/OL].[2019-01-15]. https://arxiv.org/pdf/1312.4400.pdf. [10] SZEGEDY C, LIU W, JIA Y Q, et al. Going Deeper with Convolutions // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015.DOI: 10.1109/CVPR.2015.7298594. [11] DONG C, LOY C C, HE K M, et al. Image Super-Resolution Using Deep Convolutional Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295-307. [12] DONG C, LOY C C, TANG X O. Accelerating the Super-Resolution Convolutional Neural Network[C/OL]. [2019-01-15]. https://arxiv.org/pdf/1608.00367.pdf. [13] KIM J, LEE J K, LEE K M. Accurate Image Super-Resolution Using Very Deep Convolutional Networks // Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington, USA: IEEE, 2016: 1646-1654. [14] KIM J, LEE J K, LEE K M. Deeply-Recursive Convolutional Network for Image Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington, USA: IEEE, 2016: 1637-1645. [15] YANG W H, FENG J S, YANG J C, et al. Deep Edge Guided Recurrent Residual Learning for Image Super-Resolution. IEEE Transactions on Image Processing, 2017, 26(12): 5895-5907. [16] ZENG K, YU J, WANG R X, et al. Coupled Deep Autoencoder for Single Image Super-Resolution. IEEE Transactions on Cybernetics, 2017, 47(1): 27-37. [17] YANG J C, WANG Z W, LIN Z, et al. Coupled Dictionary Trai-ning for Image Super-Resolution.IEEE Transactions on Image Processing, 2012, 21(8): 3467-3478. [18] ZEYDE R, ELAD M, PROTTER M. On Single Image Scale-up Using Sparse-Representations // Proc of the International Confe-rence on Curves and Surfaces.Berlin, Germany: Springer, 2010: 711-730. [19] ROMANO Y, PROTTER M, ELAD M. Single Image Interpolation via Adaptive Nonlocal Sparsity-Based Modeling. IEEE Transactions on Image Processing, 2014, 23(7): 3085-3098. [20] DONG W S, ZHANG L, SHI G M, et al. Nonlocally Centralized Sparse Representation for Image Restoration. IEEE Transactions on Image Processing, 2013, 22(4): 1620-1630. [21] SONG H H, LIU Q S, WANG G J, et al. Spatiotemporal Satellite Image Fusion Using Deep Convolutional Neural Networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(3): 821-829. [22] ZEILER M D, KRISHNAN D, TAYLOR G W, et al. Deconvolutional Networks // Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington, USA: IEEE, 2010: 2528-2535. [23] CLEVERT D, UNTERTHINER T, HOCHREITER S. Fast and Accurate Deep Network Learning by Exponential Linear Units(ELUs)[C/OL]. [2019-01-15]. https://arxiv.org/pdf/1511.07289.pdf. [24] KINGMA D P, BA J L. Adam: A Method for Stochastic Optimization[C/OL].[2019-01-15].https://arxiv.org/pdf/1412.6980.pdf. [25] GLOROT X, BENGIO Y. Understanding the Difficulty of Training Deep Feedforward Neural Networks // Proc of the International Conference on Artificial Intelligence and Statistics.New York, USA: ACM, 2010: 249-256. [26] GEVAERT C, GARCIA-HARO F. A Comparison of StarFM and an Unmixing-Based Algorithm for Landsat and MODIS Data Fusion. Remote Sensing of Environment, 2015, 156: 34-44. [27] ALPARONE L, BARONTI S, GARZELLI A, et al. A Global Qua-lity Measurement of Pan-Sharpened Multispectral Imagery. IEEE Geoscience and Remote Sensing Letters, 2004, 1(4): 313-317.
2019, Vol. 32 
