Hostname: page-component-848d4c4894-nmvwc Total loading time: 0 Render date: 2024-06-25T15:24:54.017Z Has data issue: false hasContentIssue false
  • English
  • Français

Attention-guided lightweight generative adversarial network for low-light image enhancement in maritime video surveillance

Published online by Cambridge University Press:  30 August 2022

Ryan Wen Liu Open the ORCID record for Ryan Wen Liu [Opens in a new window] ,
Nian Liu ,
Yanhong Huang  and
Yu Guo
Ryan Wen Liu
Affiliation:
Hubei Key Laboratory of Inland Shipping Technology, School of Navigation, Wuhan University of Technology, Wuhan, China Chongqing Research Institute, Wuhan University of Technology, Chongqing, China Hainan Institute, Wuhan University of Technology, Sanya, China
Nian Liu
Affiliation:
Hubei Key Laboratory of Inland Shipping Technology, School of Navigation, Wuhan University of Technology, Wuhan, China
Yanhong Huang*
Affiliation:
Hubei Key Laboratory of Inland Shipping Technology, School of Navigation, Wuhan University of Technology, Wuhan, China
Yu Guo*
Affiliation:
Hubei Key Laboratory of Inland Shipping Technology, School of Navigation, Wuhan University of Technology, Wuhan, China
*
*Corresponding authors. E-mails: yhhuang@whut.edu.cn; yuguo@whut.edu.cn
*Corresponding authors. E-mails: yhhuang@whut.edu.cn; yuguo@whut.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

Benefiting from video surveillance systems that provide real-time traffic conditions, automatic vessel detection has become an indispensable part of the maritime surveillance system. However, high-level vision tasks generally rely on high-quality images. Affected by the imaging environment, maritime images taken under poor lighting conditions easily suffer from heavy noise and colour distortion. Such degraded images may interfere with the analysis of maritime video by regulatory agencies, such as vessel detection, recognition and tracking. To improve the accuracy and robustness of detection accuracy, we propose a lightweight generative adversarial network (LGAN) to enhance maritime images under low-light conditions. The LGAN uses an attention mechanism to locally enhance low-light images and prevent overexposure. Both mixed loss functions and local discriminator are then adopted to reduce loss of detail and improve image quality. Meanwhile, to satisfy the demand for real-time enhancement of low-light maritime images, model compression strategy is exploited to enhance images efficiently while reducing the network parameters. Experiments on synthetic and realistic images indicate that the proposed LGAN can effectively enhance low-light images with better preservation of detail and visual quality than other competing methods.

Keywords

maritime surveillancevessel traffic servicelow-light image enhancementvessel detectiongenerative adversarial network
Type
Research Article
Information
The Journal of Navigation , Volume 75 , Issue 5 , September 2022 , pp. 1100 - 1117
Check for updates
Copyright
Copyright © The Author(s), 2022. Published by Cambridge University Press on behalf of The Royal Institute of Navigation

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bochkovskiy, A., Wang, C. Y. and Liao, H. Y. M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv :2004.10934.Google Scholar
Cai, J., Gu, S. and Zhang, L. (2018). Learning a deep single image contrast enhancer from multi-exposure images. IEEE Transactions on Image Processing, 27(4), 2049–2062.CrossRefGoogle Scholar
Chen, X., Ling, J., Wang, S., Yang, Y., Luo, L. and Yan, Y. (2021). Ship detection from coastal surveillance videos via an ensemble canny-Gaussian-morphology framework. The Journal of Navigation, 74(6), 1252–1266.CrossRefGoogle Scholar
Choi, D. H., Jang, I. H., Kim, M. H. and Kim, N. C. (2008). Color Image Enhancement Using Single-Scale Retinex Based on an Improved Image Formation Model. Proceedings of European Signal Processing Conference, Lausanne, Switzerland.Google Scholar
Fu, X., Zeng, D., Huang, Y., Zhang, X. P. and Ding, X. (2016) A Weighted Variational Model for Simultaneous Reflectance and Illumination Estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.CrossRefGoogle Scholar
Guo, X., Li, Y. and Ling, H. (2016). LIME: Low-light image enhancement via illumination map estimation. IEEE Transactions on Image Processing, 26(2), 982–993.CrossRefGoogle Scholar
Guo, C., Li, C., Guo, J., Loy, C. C., Hou, J., Kwong, S. and Cong, R. (2020). Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.CrossRefGoogle Scholar
Guo, Y., Lu, Y. and Liu, R. W. (2022). Lightweight deep network-enabled real-time low-visibility enhancement for promoting vessel detection in maritime video surveillance. The Journal of Navigation, 75(1), 230–250.CrossRefGoogle Scholar
Huang, Z., Hu, Q., Mei, Q., Yang, C. and Wu, Z. (2021). Identity recognition on waterways: A novel ship information tracking method based on multimodal data. The Journal of Navigation, 74(6), 1336–1352.CrossRefGoogle Scholar
Ignatov, A., Kobyshev, N., Timofte, R., Vanhoey, K. and Van Gool, L. (2017). DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.CrossRefGoogle Scholar
Jiang, Y., Gong, X., Liu, D., Cheng, Y., Fang, C., Shen, X., Yang, J., Zhou, P. and Wang, Z. (2021). EnlightenGAN: Deep light enhancement without paired supervision. IEEE Transactions on Image Processing, 30, 2340–2349.CrossRefGoogle ScholarPubMed
Jobson, D. J., Rahman, Z. U. and Woodell, G. A. (1997a). A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Transactions on Image Processing, 6(7), 965–976.CrossRefGoogle ScholarPubMed
Jobson, D. J., Rahman, Z. U. and Woodell, G. A. (1997b). Properties and performance of a center/surround retinex. IEEE Transactions on Image Processing, 6(3), 451–462.CrossRefGoogle ScholarPubMed
Kim, J. Y., Kim, L. S. and Hwang, S. H. (2001). An advanced contrast enhancement using partially overlapped sub-block histogram equalization. IEEE Transactions on Circuits and Systems for Video Technology, 11(4), 475–484.Google Scholar
Kim, W., Lee, R., Park, M. and Lee, S. H. (2019). Low-light image enhancement based on maximal diffusion values. IEEE Access, 7, 129150–129163.CrossRefGoogle Scholar
Land, E. H. (1964). The retinex. American Scientist, 52(2), 247–264.Google Scholar
Li, C., Guo, J., Porikli, F. and Pang, Y. (2018). Lightennet: A convolutional neural network for weakly illuminated image enhancement. Pattern Recognition Letters, 104, 15–22.CrossRefGoogle Scholar
Lin, H. and Shi, Z. (2014). Multi-scale retinex improvement for nighttime image enhancement. Optik, 125(24), 7143–7148.CrossRefGoogle Scholar
Liu, R. W., Guo, Y., Lu, Y., Chui, K. T. and Gupta, B. B. (2022). Deep network-enabled haze visibility enhancement for visual IoT-driven intelligent transportation systems. IEEE Transactions on Industrial Informatics. [online]: doi:10.1109/TII.2022.3170594.Google Scholar
Liu, R. W., Yuan, W., Chen, X. and Lu, Y. (2021). An enhanced CNN-enabled learning method for promoting ship detection in maritime surveillance system. Ocean Engineering, 235, 109435.CrossRefGoogle Scholar
Lore, K. G., Akintayo, A. and Sarkar, S. (2017). LLNet: A deep autoencoder approach to natural low-light image enhancement. Pattern Recognition, 61, 650–662.CrossRefGoogle Scholar
Lu, Y., Guo, Y., Liu, R. W. and Ren, W. (2022). MTRBNet: Multi-branch topology residual block-based network for low-light enhancement. IEEE Signal Processing Letters, 29, 1127–1131.CrossRefGoogle Scholar
Ma, J., Fan, X., Ni, J., Zhu, X. and Xiong, C. (2017). Multi-scale retinex with color restoration image enhancement based on Gaussian filtering and guided filtering. International Journal of Modern Physics B, 31(16-19), 1744077.CrossRefGoogle Scholar
Mittal, A., Moorthy, A. K. and Bovik, A. C. (2012a). No-reference image quality assessment in the spatial domain. IEEE Transactions on Image Processing, 21(12), 4695–4708.CrossRefGoogle ScholarPubMed
Mittal, A., Soundararajan, R. and Bovik, A. C. (2012b). Making a “completely blind” image quality analyzer. IEEE Signal Processing Letters, 20(3), 209–212.CrossRefGoogle Scholar
Nie, X., Yang, M. and Liu, R. W. (2019). Deep Neural Network-Based Robust Ship Detection Under Different Weather Conditions. Proceedings of the IEEE Intelligent Transportation Systems Conference, Auckland, New Zealand.CrossRefGoogle Scholar
Ooi, C. H. and Isa, N. A. M. (2010). Quadrants dynamic histogram equalization for contrast enhancement. IEEE Transactions on Consumer Electronics, 56(4), 2552–2559.CrossRefGoogle Scholar
Petro, A. B., Sbert, C. and Morel, J. M. (2014). Multiscale retinex. Image Processing On Line, 4, 71–88.CrossRefGoogle Scholar
Pizer, S. M., Amburn, E. P., Austin, J. D., Cromartie, R., Geselowitz, A., Greer, T., ter Haar Romeny, B., Zimmerman, J. B. and Zuiderveld, K. (1987). Adaptive histogram equalization and its variations. Computer Vision, Graphics, and Image Processing, 39(3), 355–368.CrossRefGoogle Scholar
Reza, A. M. (2004). Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement. Journal of VLSI Signal Processing Systems for Signal, Image and Video Technology, 38(1), 35–44.CrossRefGoogle Scholar
Senthilkumaran, N. and Thimmiaraja, J. (2014). Histogram Equalization for Image Enhancement Using MRI Brain Images. World Congress on Computing and Communication Technologies, Trichirappalli, India.CrossRefGoogle Scholar
Shao, Z., Wu, W., Wang, Z., Du, W. and Li, C. (2018). Seaships: A large-scale precisely annotated dataset for ship detection. IEEE Transactions on Multimedia, 20(10), 2593-2604.CrossRefGoogle Scholar
Veluchamy, M. and Subramani, B. (2019). Image contrast and color enhancement using adaptive gamma correction and histogram equalization. Optik, 183, 329–337.CrossRefGoogle Scholar
Wang, Z. and Bovik, A. C. (2009). Mean squared error: Love it or leave it? A new look at signal fidelity measures. IEEE Signal Processing Magazine, 26(1), 98–117.CrossRefGoogle Scholar
Wang, Z., Bovik, A. C., Sheikh, H. R. and Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.CrossRefGoogle ScholarPubMed
Wang, S., Zheng, J., Hu, H. M. and Li, B. (2013). Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Transactions on Image Processing, 22(9), 3538–3548.CrossRefGoogle ScholarPubMed
Wang, W., Wu, X., Yuan, X. and Gao, Z. (2020). An experiment-based review of low-light image enhancement methods. IEEE Access, 8, 87884–87917.CrossRefGoogle Scholar
Wei, C., Wang, W., Yang, W. and Liu, J. (2018). Deep retinex decomposition for low-light enhancement. arXiv:1808.04560.Google Scholar
Xu, K., Yang, X., Yin, B. and Lau, R. W. (2020). Learning to Restore Low-Light Images via Decomposition-and-Enhancement. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.CrossRefGoogle Scholar
Zhao, L. and Shi, G. (2019). Maritime anomaly detection using density-based clustering and recurrent neural network. The Journal of Navigation, 72(4), 894–916.CrossRefGoogle Scholar