Abstract Due to the phenomena of light absorption and scattering, as well as the presence of small particles in underwater environment, underwater images suffer from the problems of color imbalance and detail distortion. To address this issue, an underwater image enhancement network based on visual multi-head attention and skip-layer whitening is proposed in this paper. A hierarchical architecture is adopted, feature extraction is performed by the encoding path and image reconstruction is carried out by the decoding path. The main components of the encoding and decoding paths are visual multi-head self-attention blocks. Instance whitening is applied to shallow features. The features of shallow layer after instance whitening are embedded into the features of deep layer by skip-layer connection as skip-layer whitening path. Content loss and structure loss are employed in the training process of the proposed network. The comparative experiment on benchmark underwater image datasets demonstrates the effectiveness of visual multi-head self-attention and instance whitening for underwater enhancement task, both quantitatively and visually.
Fund:National Natural Science Foundation of China(No.62172090), Start-up Research Fund of Southeast University(No.RF1028623097)
Corresponding Authors:
ZHANG Jun, Ph.D., professor. His research interests include computer vision and pattern recognition.
About author:: CONG Xiaofeng, Ph.D. candidate. His research interests include underwater image enhancement and image dehazing.GUI Jie, Ph.D., professor. His research interests include generative algorithm, image dehazing and self-supervised learning.HE Lei, Ph.D., associate professor. His research interests include cyberspace security and endogenous safety.
[1] 蔡达,范保杰. 基于空间特征选择的水下目标检测方法. 信息与控制, 2022, 51(2): 214-222. (CAI D, FAN B J. Spatial Feature Selection for Underwater Object Detection. Information and Control, 2022, 51(2): 214-222.) [2] BERMAN D, LEVY D, AVIDAN S, et al. Underwater Single Ima-ge Color Restoration Using Haze-Lines and a New Quantitative Dataset. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(8): 2822-2837. [3] YANG H H, HUANG K C, CHEN W T. LAFFNet: A Lightweight Adaptive Feature Fusion Network for Underwater Image Enhancement // Proc of the IEEE International Conference on Robotics and Automation. Washington, USA: IEEE, 2021: 685-692. [4] WEI Y W, ZHENG Z R, JIA X Y. UHD Underwater Image Enhancement via Frequency-Spatial Domain Aware Network // Proc of the Asian Conference on Computer Vision. Berlin, Germany: Springer, 2022: 21-36. [5] 陈学磊,张品,权令伟,等. 融合深度学习与成像模型的水下图像增强算法. 计算机工程, 2022, 48(2): 243-249. (CHEN X L, ZHANG P, QUAN L W, et al. Underwater Image Enhancement Algorithm Combining Deep Learning and Image Formation Model. Computer Engineering, 2022, 48(2): 243-249) [6] UPLAVIKAR P, WU Z Y, WANG Z Y. All-in-One Underwater Ima-ge Enhancement Using Domain-Adversarial Learning[C/OL]. [2023-2-16].https://arxiv.org/pdf/1905.13342.pdf. [7] ANWAR S, LI C Y. Diving Deeper into Underwater Image Enhancement: A Survey. Signal Processing: Image Communication, 2020, 89. DOI: 10.1016/j.image.2020.115978. [8] PENG Y T, COSMAN P C. Underwater Image Restoration Based on Image Blurriness and Light Absorption. IEEE Transactions on Image Processing, 2017, 26(4): 1579-1594. [9] SONG W, WANG Y, HUANG D M, et al. A Rapid Scene Depth Estimation Model Based on Underwater Light Attenuation Prior for Underwater Image Restoration // Proc of the Pacific-Rim Conference on Multimedia. Berlin, Germany: Springer, 2018: 678-688. [10] CARLEVARIS-BIANCO N, MOHAN A, EUSTICE R M. Initial Results in Underwater Single Image Dehazing[C/OL]. [2023-2-16].https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5664428. [11] SONG W, WANG Y, HUANG D M, et al. Enhancement of Underwater Images with Statistical Model of Background Light and Optimization of Transmission Map. IEEE Transactions on Broadcasting, 2020, 66(1): 153-169. [12] LIU P, WANG G Y, QI H, et al. Underwater Image Enhancement with a Deep Residual Framework. IEEE Access, 2019, 7: 94614-94629. [13] ISLAM M J, XIA Y Y, SATTAR J. Fast Underwater Image Enhancement for Improved Visual Perception. IEEE Robotics and Automation Letters, 2020, 5(2): 3227-3234. [14] FABBRI C, ISLAM M J, SATTAR J. Enhancing Underwater Ima-gery Using Generative Adversarial Networks // Proc of the IEEE International Conference on Robotics and Automation. Washington, USA: IEEE, 2018: 7159-7165. [15] MA Z Y, OH C. A Wavelet-Based Dual-Stream Network for Underwater Image Enhancement // Proc of the IEEE International Conference on Acoustics, Speech and Signal Processing. Washington, USA: IEEE, 2022: 2769-2773. [16] JAMADANDI A, MUDENAGUDI U. Exemplar-Based Underwater Image Enhancement Augmented by Wavelet Corrected Transforms // Proc of the IEEE/CVF Conference on Computer Vision and Pa-ttern Recognition Workshops. Washington, USA: IEEE, 2019: 11-17. [17] YAN X H, QIN W Q, WANG Y F, et al. Attention-Guided Dynamic Multi-branch Neural Network for Underwater Image Enhancement. Knowledge-Based Systems, 2022, 258. DOI: 10.1016/j.knosys.110041. [18] TANG Y, IWAGUCHI T, KAWASAKI H, et al. AutoEnhancer: Transformer on U-Net Architecture Search for Underwater Image Enhancement // Proc of the Asian Conference on Computer Vision. Berlin, Germany: Springer, 2022: 1403-1420. [19] CHEN Y W, PEI S C. Domain Adaptation for Underwater Image Enhancement via Content and Style Separation. IEEE Access, 2022, 10: 90523-90534. [20] FU Z Q, LIN X P, WANG W, et al. Underwater Image Enhancement via Learning Water Type Desensitized Representations // Proc of the IEEE International Conference on Acoustics, Speech and Signal Processing. Washington, USA: IEEE, 2022: 2764-2768. [21] NAIK A, SWARNAKAR A, MITTAL K. Shallow-UWnet: Compressed Model for Underwater Image Enhancement(Student Abstract) // Proc of the AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI, 2021: 15853-15854. [22] LI C Y, GUO C L, REN W Q, et al. An Underwater Image Enhancement Benchmark Dataset and Beyond. IEEE Transactions on Image Processing, 2019, 29: 4376-4389. [23] 田永林,王雨桐,王建功,等. 视觉Transformer研究的关键问题:现状及展望. 自动化学报, 2022, 48(4): 957-979. (TIAN Y L, WANG Y T, WANG J G, et al. Key Problems and Progress of Vision Transformers: The State of the Art and Prospects. Acta Automatica Sinica, 2022, 48(4): 957-979.) [24] VASWANI A, SHAZEER N, PARMAR N, et al. Attention Is All You Need // Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2017: 6000-6010. [25] LIU Z, LIN Y Y, CAO Y, et al. Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2021: 9992-10002. [26] 薛峰,洪自坤,李书杰,等. 基于Vision Transformer的中文唇语识别. 模式识别与人工智能, 2022, 35(12): 1111-1121. (XUE F, HONG Z K, LI S J, et al. Chinese Lipreading Network Based on Vision Transformer. Pattern Recognition and Artificial Intelligence, 2022, 35(12): 1111-1121.) [27] 张玉朋,李香菊,李超,等. 基于Transformer与异质图神经网络的新闻推荐模型. 模式识别与人工智能, 2022, 35(9): 839-848. (ZHANG Y P, LI X J, LI C, et al. News Recommendation Model Based on Transformer and Heterogenous Graph Neural Network. Pattern Recognition and Artificial Intelligence, 2022, 35(9): 839-848.) [28] BOUDIAF A, GUO Y H, GHIMIRE A, et al. Underwater Image Enhancement Using Pre-trained Transformer // Proc of the 21st International Conference on Image Analysis and Processing. New York, USA: ACM, 2022: 480-488. [29] CHEN H T, WANG Y H, GUO T Y, et al. Pre-trained Image Processing Transformer // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 12294-12305. [30] PARK N, KIM S. How Do Vision Transformers Work?[C/OL]. [2023-2-16].https://arxiv.org/pdf/2202.06709.pdf. [31] PAN X G, ZHAN X H, SHI J P, et al. Switchable Whitening for Deep Representation Learning // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 1863-1871. [32] SONG Y D, HE Z Q, QIAN H, et al. Vision Transformers for Single Image Dehazing. IEEE Transactions on Image Processing, 2023, 32: 1927-1941. [33] WANG Z, BOVIK A C, SHEIKH H R, et al. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600-612. [34] PANETTA K, GAO C, AGAIAN S. Human-Visual-System-Ins-pired Underwater Image Quality Measures. IEEE Journal of Oceanic Engineering, 2016, 41(3): 541-551.
2023, Vol. 36 
