|
|
|
| Underwater Image Enhancement Network Based on Octave Convolution with Physical Model Constraints |
| WU Cheng1, LIU Xiaohua2, TANG Guijin1 |
1. School of Communications and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing 210003;
2. School of Digital Media and Design Arts, Nanjing University of Posts and Telecommunications, Nanjing 210023 |
|
|
|
|
Abstract In underwater image enhancement tasks,deep learning struggles with the lack of physical interpretability and the spatial redundancy in feature representation. To address these issues, an underwater image enhancement network based on Octave convolution with physical model constraints（OCPMNet） is proposed. First, an octave feature extraction block（OFEB） is designed to explicitly decompose the input image into a high-frequency branch and a low-frequency branch via octave convolution. The receptive field of the low-frequency branch is expanded through downsampling to capture global degradation, and meanwhile the computational redundancy is effectively reduced. Then, a multi-scale background light module （MBLM） is constructed to extract deep low-frequency features from the bottleneck layer of the network. Three branches are utilized to extract local, neighbor and global background light information, and the information is fused to obtain an estimated map of the ambient background light. Finally, a physics-constrained dual-domain restoration module（PDRM） is introduced. Octave convolution is adopted to fuse terminal features, yielding the estimations of the direct transmission map and the backscatter transmission map. Moreover, an initial enhanced image is reconstructed under the constraints of the Sea-Thru physical model with the background light estimated by MBLM. Subsequently, residual correction is performed simultaneously in both spatial and frequency domains to compensate for the fitting bias of the physical model. Experiments on multiple datasets demonstrate the superiority of OCPMNet in terms of objective metrics and subjective image quality. Furthermore, OCPMNet shows strong real-time processing capability with engineering application value.
|
|
Received: 16 January 2026
|
|
|
| Fund:University-Industry Cooperative Education Program of Ministry of Education of China（No.231101019275515，231107604232536），Key Project of Educational Science Planning of Jiangsu Province（No.B/2023/01/211），Postgraduate Research & Practice Innovation Program of Jiangsu Province（No.SJCX25_0304） |
|
Corresponding Authors:
TANG Guijin, Ph.D., professor. His research interests include ima-ge processing, video analysis, and multimedia communication.
|
| About author:: About Author:WU Cheng, Master student. His research interests include image processing and underwater image enhancement. LIU Xiaohua, Ph.D., senior experimentalist. Her research interests include image restoration, image reconstruction, and pattern recognition. |
|
|
|
[1] WANG M J, ZHANG K K, WEI H A, et al. Underwater Image Quality Optimization: Researches, Challenges, and Future Trends. Image and Vision Computing, 2024, 146. DOI: 10.1016/j.imavis.2024.104995.
[2] MOGHIMI M K, MOHANNA F.Real-Time Underwater Image Enhancement: A Systematic Review. Journal of Real-Time Image Processing, 2021, 18(5): 1509-1525.
[3] AKKAYNAK D, TREIBITZ T. A Revised Underwater Image Formation Model // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 6723-6732.
[4] PIZER S M, AMBURN E P, AUSTIN J D, et al. Adaptive Histogram Equalization and Its Variations. Computer Vision, Graphics, and Image Processing, 1987, 39(3): 355-368.
[5] CHEN Y P, FAN H Q, XU B, et al. Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 3434-3443.
[6] AKKAYNAK D, TREIBITZ T. Sea-Thru: A Method for Removing Water from Underwater Images // Proc of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 1682-1691.
[7] XIANG D, WANG H H, HE D Y, et al. Research on Histogram Equalization Algorithm Based on Optimized Adaptive Quadruple Segmentation and Cropping of Underwater Image(AQSCHE). IEEE Access, 2023, 11: 69356-69365.
[8] ZHANG D, HE Z X, ZHANG X H, et al. Underwater Image Enhancement via Multi-scale Fusion and Adaptive Color-Gamma Co-rrection in Low-Light Conditions. Engineering Applications of Artificial Intelligence, 2023, 126(C). DOI: 10.1016/j.engappai.2023.106972.
[9] JAFFE J S.Computer Modeling and the Design of Optimal Underwater Imaging Systems. IEEE Journal of Oceanic Engineering, 1990, 15(2): 101-111.
[10] PENG Y T, COSMAN P C.Underwater Image Restoration Based on Image Blurriness and Light Absorption. IEEE Transactions on Image Processing, 2017, 26(4): 1579-1594.
[11] SONG W, WANG Y, HUANG D M, et al. A Rapid Scene Depth Estimation Model Based on Underwater Light Attenuation Prior for Underwater Image Restoration // Proc of the 19th Pacific-Rim Conference on Multimedia. Berlin, Germany: Springer, 2018: 678-688.
[12] LI C Y, GUO C L, REN W Q, et al. An Underwater Image Enhancement Benchmark Dataset and Beyond. IEEE Transactions on Image Processing, 2020, 29: 4376-4389.
[13] PENG W Z, ZHOU C H, HU R Z, et al. RAUNE-Net: A Resi-dual and Attention-Driven Underwater Image Enhancement Method // Proc of the 20th International Forum on Digital TV and Wireless Multimedia Communications. Berlin, Germany: Springer, 2024: 15-27.
[14] REN T D, XU H Y, JIANG G Y, et al. Reinforced Swin-Convs Transformer for Simultaneous Underwater Sensing Scene Image Enhancement and Super-Resolution. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60. DOI: 10.1109/TGRS.2022.3205061
[15] PENG L T, ZHU C L, BIAN L H.U-Shape Transformer for Underwater Image Enhancement. IEEE Transactions on Image Processing, 2023, 32: 3066-3079.
[16] FABBRI C, ISLAM M J, SATTAR J. Enhancing Underwater Ima-gery Using Generative Adversarial Networks // Proc of the IEEE International Conference on Robotics and Automation. Washington, USA: IEEE, 2018: 7159-7165.
[17] CHEN Y H, YAN Q Y, JIN S G, et al. DiffWater: A Conditional Diffusion Model for Estimating Surface Water Fraction Using CyGNSS Data. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63. DOI: 10.1109/TGRS.2025.3564612.
[18] ISLAM M J, XIA Y Y, SATTAR J.Fast Underwater Image Enhancement for Improved Visual Perception. IEEE Robotics and Automation Letters, 2020, 5(2): 3227-3234.
[19] CONG R M, YANG W Y, ZHANG W, et al. PUGAN: Physical Model-Guided Underwater Image Enhancement Using GAN with Dual-Discriminators. IEEE Transactions on Image Processing, 2023, 32: 4472-4485.
[20] BACH N G, TRAN C M, KAMIOKA E, et al. Underwater Image Enhancement with Physical-Based Denoising Diffusion Implicit Models. Journal of Image and Graphics, 2025, 13(3): 213-230.
[21] ZHAO C, CAI W L, DONG C Y, et al. Wavelet-Based Fourier Information Interaction with Frequency Diffusion Adjustment for Underwater Image Restoration // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2024: 8281-8291.
[22] LINDEBERG T. Scale-Space Theory in Computer Vision. Berlin, Germany: Springer, 2013.
[23] ZHU Z Q, LUO Y Q, QI G Q, et al. Remote Sensing Image Defogging Networks Based on Dual Self-Attention Boost Residual Octave Convolution. Remote Sensing, 2021, 13(16). DOI: 10.3390/rs13163104.
[24] WANG M H, LIAO L, HUANG D, et al. Frequency and Content Dual Stream Network for Image Dehazing. Image and Vision Computing, 2023, 139. DOI: 10.1016/j.imavis.2023.104820.
[25] HU J, SHEN L, ALBANIE S, et al. Squeeze-and-Excitation Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023.
[26] LI H Y, LI J J, WANG W.A Fusion Adversarial Underwater Ima-ge Enhancement Network with a Public Test Dataset[C/OL]. [2025-12-16].https://arxiv.org/pdf/1906.06819.
[27] HE K M, ZHANG X Y, REN S Q, et al. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2015: 1026-1034.
[28] ZHANG R, ISOLA P, EFROS A A, et al. The Unreasonable Effec-tiveness of Deep Features as a Perceptual Metric // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 586-595.
[29] YANG M, SOWMYA A.An Underwater Color Image Quality Evaluation Metric. IEEE Transactions on Image Processing, 2015, 24(12): 6062-6071.
[30] PANETTA K, GAO C, AGAIAN S.Human-Visual-System-Inspired Underwater Image Quality Measures. IEEE Journal of Oceanic Engineering, 2016, 41(3): 541-551.
[31] KE J J, WANG Q F, WANG Y L, et al. MUSIQ: Multi-scale Ima-ge Quality Transformer // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2021: 5128-5137.
[32] WANG J Y, CHAN K C K, LOY C C. Exploring CLIP for Asse-ssing the Look and Feel of Images. Proceedings of the AAAI Conference on Artificial Intelligence, 2023, 37(2): 2555-2563. |
|
|
|
2026, Vol. 39 
