Skip to main content Accessibility help
×
Home

Graph-Based Registration and Blending for Undersea Image Stitching

  • Xu Yang (a1), Zhi-Yong Liu (a1), Hong Qiao (a1), Jian-Hua Su (a1), Da-Xiong Ji (a2), Ai-Yun Zang (a3) and Hai Huang (a4)...

Summary

Image stitching is important for the perception and manipulation of undersea robots. In spite of a well-developed technique, it is still challenging for undersea images because of their inevitable appearance ambiguity caused by the limited light in the undersea environment, and local disturbance caused by moving objects, ocean current, etc. To get a clean and stable background panorama in the undersea environment, this paper proposes an undersea image-stitching method by introducing graph-based registration and blending procedures. Specifically, in the registration procedure, matching the features in each undersea image pair is formulated and solved by graph matching, to incorporate the structural information between features. In the blending procedure, an energy function on the indirect graph Markov random field is proposed, which takes both image consistency and neighboring consistency into consideration. Coincidentally, both graph matching and energy minimization can be mathematically formulated by integer quadratic programming problems with different constraints; the recently proposed graduated nonconvexity and concavity procedure is used to optimize both problems. Experiments on both synthetic images and real-world undersea images witness the effectiveness of the proposed method.

Copyright

Corresponding author

*Corresponding author. E-mail: haihus@163.com

References

Hide All
1.Bay, H., Tuytelaars, T. and Van Gool, L., “SURF: Speeded Up Robust Features,” Proceedings of the 7th European Conference on Computer Vision, Austria (2006) pp. 404417.
2.Belongie, S. and Malik, J., “Matching with Shape Contexts,” Proceedings of the IEEEWorkshop on Contentbased Access of Image and Video Libraries, Hilton Head Island, SC, USA (2000) pp. 2026.
3.Boyd, S. and Vandenberghe, L., Convex Optimization (Cambridge University Press, Cambridge, UK, 2004).
4.Boykov, Y., Veksler, O. and Zabih, R., “Fast approximate energy minimization via graph cuts,” IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 12221239 (2001).
5.Brown, M. and Lowe, D. G., “Automatic panoramic image stitching using invariant features,” Int’l. J. Comput. Vision. 74(1), 5973 (2007).
6.Calonder, M., Lepetit, V., Strecha, C. and Fua, P., “BRIEF: Binary Robust Independent Elementary Features,” European Conference on Computer Vision (Springer, Berlin, Heidelberg, 2010) pp. 778792.
7.Cour, T. and Shi, J. B., “Solving Markov Random Fields with Spectral Relaxation,” Proceedings of the International Conference on Artificial Intelligence and Statistics, San Juan, Puerto Rico (2007)
8.Davis, J., “Mosaics of Scenes with Moving Objects,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Santa Barbara, CA, USA (1998) pp. 354360.
9.Egozi, A., Keller, Y. and Guterman, H., “A probabilistic approach to spectral graph matching,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 1827 (2013).
10.Elibol, A., Garcia, R. and Gracias, N., “A new global alignment approach for underwater optical mapping,” Ocean Eng. 38(10), 12071219 (2011).
11.Ferreira, F., Veruggio, G., Caccia, M., Zereik, E. and Bruzzone, G., “A Real-Time Mosaicking Algorithm Using Binary Features for ROVs,” Proceedings of the Mediterranean Conference on Control and Automation, Chania (2013) pp. 12671273.
12.Finlayson, G., Hordley, S., Schaefer, G. and Tian, G. Y., “Illuminant and device invariant colour using histogram equalisation,” Pattern Recogn. 38(2), 179190 (2005).
13.Fischler, M. A. and Bolles, R. C., “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM. 24(6), 381395 (1981).
14.Frank, M. and Wolfe, P., “An algorithm for quadratic programming,” Naval Res. Logis. Q. 3(1–2), 95110 (1956).
15.Garcia-Fidalgo, E., Ortiz, A., Bonnin-Pascual, F. and Company, J. P., “Fast Image Mosaicing Using Incremental Bags of Binary Words,” Proceedings of the IEEE International Conference on Robotics and Automation, Stockholm, Sweden (2016) pp. 11741180.
16.Jaggi, M., “Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization,” Proceedings of the International Conference on Machine Learning, Atlanta, GA, USA (2013) pp. 427435.
17.Kolmogorov, V., “Convergent tree-reweighted message passing for energy minimization,” IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 15681583 (2006).
18.Kolmogorov, V. and Rother, C., “Minimizing nonsubmodular functions with graph cuts a review,” IEEE Trans. Pattern Anal. Mach. Intell. 29(7), 12741279 (2007).
19.Leone, A., Distante, C., Mastrolia, A. and Indiveri, G.,” A Fully Automated Approach for Underwater Mosaicking,” OCEANS (2006) pp. 16.
20.Leordeanu, M. and Hebert, M., “A Spectral Technique for Correspondence Problems Using Pairwise Constraints,” Proceedings of the IEEE International Conference on Computer Vision, Beijing, China (2005) pp. 14821489.
21.Leordeanu, M. and Hebert, M., “Efficient MAP Approximation for Dense Energy Functions,” Proceedings of the International Conference on Machine Learning, Pittsburgh, Pennsylvania, USA (2006).
22.Li, C., Liu, Z. Y., Yang, X., Qiao, H. and Su, J. H., “Stitching contaminated images,” Neurocomputing 214, 829836 (2016).
23.Liu, Z. Y. and Qiao, H., “GNCCP - graduated nonconvexity and concavity procedure,” IEEE Trans. Pattern Anal. Mach. Intell. 36(6), 12581267 (2014).
24.Loera, J. D., Rambau, J. and Santos, F., Triangulations: Structures for Algorithms and Applications (Springer-Verlag, Berlin, Heidelberg, 2010).
25.Lowe, D. G., “Object Recognition from Local Scale-Invariant Features,” Proceedings of IEEE International Conference on Computer Vision, Kerkyra, Greece, vol. 2 (1999) pp. 11501157.
26.Mills, A. and Dudek, G., “Image stitching with dynamic elements,” Image Vision Comput. 27(10), 15931602 (2009).
27.Minka, T. P., “Old and New Matrix Algebra Useful for Statistics,” Technical report (2001).
28.Ravikumar, P. and Lafferty, J., “Quadratic Programming Relaxations for Metric Labeling and Markov Random Field Map Estimation,” Proceedings of the International Conference on Machine Learning, Pittsburgh, Pennsylvania, USA (2006) pp. 737744.
29.Sun, J., Zheng, N. N. and Shum, H. Y., “Stereo matching using belief propagation,” IEEE Trans. Pattern Anal. Mach. Intell. 25(7), 787800 (2003).
30.Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M. and Rother, C., “A comparative study of energy minimization methods for Markov random fields with smoothness-based priors,” IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 10681080 (2008).
31.Tang, C. Q., Tian, G. Y., Chen, X. T., Wu, J. B., Li, K. and Meng, H., “Infrared and visible images registration with adaptable local-global feature integration for rail inspection,” Infrared Phys. Technol. 87, 3139 (2017).
32.Tian, G. Y., Gledhill, D. and Taylor, D., “Comprehensive interest points based imaging mosaic,” Pattern Recogn. Lett. 24(9), 11711179 (2003).
33.Torr, P. H. S., “Solving Markov Random Fields Using Semi Definite Programming,” Proceedings of the International Conference on Artificial Intelligence and Statistics, Key West, Florida, USA (2003).
34.Torr, P. H. S. and Zisserman, A., “MLESAC: A new robust estimator with application to estimating image geometry,” Comput. Vision Image Understand. 78, 138156 (2000).
35.Uyttendaele, M., Eden, A. and Skeliski, R., “Eliminating Ghosting and Exposure Artifacts in Image Mosaics,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HI, USA (2001) pp. II509II516.
36.Yang, X., Liu, Z. Y., Qiao, H., Song, Y. B., Ren, S. N. and Zheng, S. W., “Underwater image matching by incorporating structural constraints,” Int’l. J. Adv. Robot. Syst. to appear, https://doi.org/10.1177/1729881417738100.

Keywords

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed