Monte Carlo Noise Removal Algorithm Based on Adversarial Generative Network
XIE Chuan1,2, WANG Yongchao1, LIN Zhijie3, ZHENG Qiulan4, QIAN Fei1, ZHAO Lei1
1.School of Computer Science and Technology, Zhejiang University, Hangzhou 310027 2.School of Information Engineering, Hangzhou Vocational and Technical College, Hangzhou 310018 3.School of Information and Electronic Engineering, Zhejiang University of Science and Technology, Hangzhou 310023 4.Institute of Health Food, Zhejiang Academy of Medical Sciences, Hangzhou 310013
Abstract To solve the problem of high frequency details loss in the existing Monte Carlo noise removal method, a Monte Carlo noise removal method based on adversarial generative network is proposed. An adversarial network structure, including the generative network of full convolution network and the discriminator network of deep convolution network, is employed to remove the Monte Carlo noise. The multi-dimensional auxiliary features, including the pixel color of the image, are added as the network input. Besides, the new loss function and local importance sampling technology based on the similarity deviation between normal vector variance and gradient size are applied to network training. Experimental results show that the proposed method achieves good quantization index in removing Monte Carlo noise and meanwhile preserves high-frequency detail features of the image.
Fund:Supported by Science and Technology Plan Project of Zhejiang Province(No.2017C33176,LGF18F020006,LGF18F020010), Doctoral Scientific Research Foundation of Zhejiang University of Science and Technology(No.170311)
Corresponding Authors:
LIN Zhijie, Ph.D., lecturer. His research interests include digital image processing,computer vision and machine learning.
About author:: XIE Chuan, master, associate professor. His research interests include object recognition, image denoising and deep learning;WANG Yongchao, master, associate professor. His research interests include image processing, 3D display and video compre-ssion;ZHENG Qiulan, master, research assistant. Her research interests include food qua-lity control and safety evaluation;QIAN Fei, master, His research interests include in-depth study, image processing and virtual reality;ZHAO Lei, Ph.D., lecturer. His research interests include deep learning, image processing and virtual reality.
[1]BURGEN H C, SCHULER C J, HARMELING S. Image Denoising: Can Plain Neural Networks Compete with BM3D? // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2012: 2392-2399. [2]JAIN V, SÉUNG H S. Natural Image Denoising with Convolutional Networks // KOLLER D, SCHUURMANS D, BENGIO Y, et al., eds. Advances in Neural Information Processing Systems 21. Cambridge, USA: The MIT Press, 2009: 769-776. [3]XIE J Y, XU L L, CHEN E H. Image Denoising and Inpainting with Deep Neural Networks // PEREIRA F, BURGES C J C, BOTTOU L, et al., eds. Advances in Neural Information Processing Systems 25. Cambridge, USA: The MIT Press, 2012: 341-349. [4]AHN B, CHO N I. Block-Matching Convolutional Neural Network for Image Denoising[J/OL]. [2018-03-24]. https://arxiv.org/pdf/1704.00524.pdf. [5]ZHANG K, ZUO W M, ZHANG L. FFDNet: Toward a Fast and Flexible Solution for CNN Based Image Denoising. IEEE Transactions on Image Processing, 2018, 27(9): 4608-4622. [6]ZHANG K, ZUO W M, CHEN Y J, et al. Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising. IEEE Transactions on Image Processing, 2017, 26(7): 3142-3155. [7]VEMULAPALLI R, TUZEL O, LIU M Y. Deep Gaussian Conditional Random Field Network: A Model-Based Deep Network for Discriminative Denoising[C/OL]. [2018-03-24]. https://arxiv.org/pdf/1511.04067.pdf. [8]DIVAKAR N, BABU R V. Image Denoising via CNNs: An Adversarial Approach // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 80-87. [9]GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al.Generative Adversarial Nets // GHAHRAMANI Z, WELLING M, CORTES C, et al., eds. Advances in Neural Information Processing Systems 27. Cambridge, USA: The MIT Press, 2014: 2672-2680. [10]XU R F, PATTANAIK S N. A Novel Monte Carlo Noise Reduction Operator. IEEE Computer Graphics and Applications, 2005, 25(2): 31-35. [11]ROUSSELLE F, KNAUS C, ZWICKER M. Adaptive Rendering with Non-local Means Filtering. ACM Transactions on Graphics, 2012, 31(6). DOI: 10.1145/2366145.2366214. [12]ROUSSELLE F, KNAUS C, ZWICKER M. Adaptive Sampling and Reconstruction Using Greedy Error Minimization. ACM Transactions on Graphics, 2011, 30(6). DOI: 10.1145/2070781.2024193. [13]KALANTARI N K, SEN P. Removing the Noise in Monte Carlo Rendering with General Image Denoising Algorithms. Computer Graphics Forum, 2013, 32(2): 93-102. [14]BAUSZAT P, EISEMANN M, EISEMANN E, et al. General and Robust Error Estimation and Reconstruction for Monte Carlo Rendering. Computer Graphics Forum, 2015, 34(2): 597-608. [15]DELBRACIO M, MUSÉP, BUADES A, et al. Boosting Monte Carlo Rendering by Ray Histogram Fusion. ACM Transactions on Graphics, 2014, 33(1). DOI: 10.1145/2532708. [16]KALANTARI N K, BAKO S, SEN P. A Machine Learning Approach for Filtering Monte Carlo Noise. ACM Transactions on Graphics, 2015, 34(4). DOI: 10.1145/2766977. [17]BOUGHIDA M, BOUBEKEUR T. Bayesian Collaborative Denoi-sing for Monte Carlo Rendering. Computer Graphics Forum, 2017, 36(4): 137-153. [18]MCCOOL M D. Anisotropic Diffusion for Monte Carlo Noise Reduction. ACM Transactions on Graphics, 1999, 18(2): 171-194. [19]ROUSSELLE F, MANZI M, ZWICKER M. Robust Denoising Using Feature and Color Information. Computer Graphics Forum, 2013, 32(7): 121-130. [20]SEN P, DARABI S. On Filtering the Noise from the Random Parameters in Monte Carlo Rendering. ACM Transactions on Gra-phics, 2012, 31(3). DOI: 10.1145/2167076.2167083. [21]MOON B, CARR N, YOON S E. Adaptive Rendering Based on Weighted Local Regression. ACM Transactions on Graphics, 2014, 33(5). DOI: 10.1145/2641762. [22]SHIRLEY P, AILA T, COHEN J, et al.A Local Image Reconstruction Algorithm for Stochastic Rendering // Proc of the Symposium on Interactive 3D Graphics and Games. New York, USA: ACM, 2011: 9-14. [23]SCHIED C, KAPHANYAN A, WYMAN C, et al.Spatiotemporal Variance-Guided Filtering: Real-Time Reconstruction for Path-Traced Global Illumination // Proc of High Performance Graphics. New York, USA: ACM, 2017. DOI: 10.1145/3105762.3105770. [24]MARA M, MCGUIRE M, BITTERLI B, et al. An Efficient Denoising Algorithm for Global Illumination[J/OL]. [2018-03-24]. https://cs.dartmouth.edu/~wjarosz/publications/mara17towards.pdf. [25]BAKO S, VOGELS T, MCWILLIAMS B, et al.Kernel-Predicting Convolutional Networks for Denoising Monte Carlo Renderings. ACM Transactions on Graphics, 2017, 36(4). DOI: 10.1145/3072959.3073708. [26]CHAITANYA C R A, KAPLANYAN A, SCHIED C, et al. Interactive Reconstruction of Noisy Monte Carlo Image Sequences Using a Recurrent Autoencoder[J/OL]. [2018-03-24]. https://research.nvidia.com/sites/default/files/publications/dnn_denoise_author.pdf. [27]FALOUTSOS C, BHAGWAT P. Declustering Using Fractals // Proc of the 2nd International Conference on Parallel and Distributed Information Systems. Washington, USA: IEEE, 1993: 18-25. [28]LEDIG C, THEIS L, HUSZÁR F, et al.Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network[C/OL]. [2018-03-24]. https://arxiv.org/pdf/1609.04802.pdf. [29]RADFORD A, METZ L, CHINTALA S. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks[J/OL]. [2018-03-24]. https://arxiv.org/pdf/1511.06434.pdf. [30]YEH R A, CHEN C, LIM T Y, et al. Semantic Image Inpainting with Deep Generative Models // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 5485-5493. [31]ISOLA P, ZHU J Y, ZHOU T H, et al. Image-to-Image Translation with Conditional Adversarial Networks[J/OL]. [2018-03-24]. https://arxiv.org/pdf/1611.07004.pdf. [32]RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional Networks for Biomedical Image Segmentation // Proc of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2015: 234-241. [33]ODENA A, DUMOULIN V, OLAH C. Deconvolution and Che-ckerboard Artifacts. Distill, 2016. DOI: 10.23915/distill.00003. [34]ZHANG H, SINDAGI V, PATEL V M. Image De-raining Using a Conditional Generative Adversarial Network[J/OL]. [2018-03-24]. https://arxiv.org/pdf/1701.05957v1.pdf. [35]ZHAO H, GALLO O, FROSIO I, et al.Loss Functions for Image Restoration with Neural Networks. IEEE Transactions on Computational Imaging, 2017, 3(1): 47-57. [36]XUE W F, MOU X Q, ZHANG L, et al.Blind Image Quality Assessment Using Joint Statistics of Gradient Magnitude and Laplacian Features. IEEE Transactions on Image Processing, 2014, 23(11): 4850-4862. [37]LI T M, WU Y T, CHUANG Y Y. Sure-Based Optimization for Adaptive Sampling and Reconstruction. ACM Transactions on Graphics, 2012, 31(6). DOI: 10.1145/2366145.2366213.
2018, Vol. 31 
