Abstract
Hull inspection is an important task to ensure sustainability of ships. To overcome the challenges of hull structure inspection in an underwater environment in an efficient way, an autonomous system for hull inspection has to be developed. In this paper, a new approach to underwater ship hull inspection is proposed. It aims at developing the basis for an end-to-end autonomous solution. The real-time aspect is an important part of this work, as it allows the operators and inspectors to receive feedback about the inspection as it happens. A reference mission plan is generated and adapted online based on the inspection findings. This is done through the processing of a multibeam forward looking sonar to estimate the pose of the hull relative to the drone. An inspection map is incrementally built in a novel way, incorporating uncertainty estimates to better represent the inspection state, quality, and observation confidence. The proposed methods are experimentally tested in real-time on real ships and demonstrate the applicability to quickly understand what has been done during the inspection.
Article PDF
Similar content being viewed by others
Data Availability
The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.
References
Hedgpeth, J.W.: Marine fouling and its prevention. Science 118(3061), 257–257 (1953). https://doi.org/10.1126/science.118.3061.257.a
Boon, B., Brennan, F., Garbatov, Y., Ji, C., Parunov, J., Rahman, T., Rizzo, C., Rouhan, A., Shin, C., Yamamoto, N.: Condition assessment of aged ships and offshore structures. In: International Ship and Offshore Structures Congress, 2, 313–365 (2009)
Mittleman, J., Swan, L.: Underwater inspection for welding and overhaul. Naval Eng. J. 105(5), 37–42 (1993). https://doi.org/10.1111/j.1559-3584.1993.tb02755.x
Lynn, D.C., Bohlander, G.S.: Performing ship hull inspections using a remotely operated vehicle. In: Oceans ’99. MTS/IEEE. Riding the Crest Into the 21st Century. Conference and Exhibition. Conference Proceedings (IEEE Cat. No.99CH37008), 2, pp. 555–5622 (1999). https://doi.org/10.1109/OCEANS.1999.804763
Vaganay, J., Elkins, M.L., Willcox, S., Hover, F.S., Damus, R.S., Desset, S., Morash, J.P., Polidoro, V.C.: Ship hull inspection by hull-relative navigation and control. In: Proceedings of OCEANS 2005 MTS/IEEE, pp. 761–7661 (2005). https://doi.org/10.1109/OCEANS.2005.1639844
Kaess, M., Johannsson, H., Englot, B., Hover, F.S., Leonard, J.J.: Towards autonomous ship hull inspection using the bluefin hauv. In: 9th International Symposium on Technology and the Mine Problem. Conference Proceedings, pp. 1–10 (2010)
Hover, F.S., Eustice, R.M., Kim, A., Englot, B., Johannsson, H., Kaess, M., Leonard, J.J.: Advanced perception, navigation and planning for autonomous in-water ship hull inspection. Int. J. Robotics Res. 31(12), 1445–1464 (2012). https://doi.org/10.1177/0278364912461059
Hong, S., Chung, D., Kim, J., Kim, Y., Kim, A., Yoon, H.K.: In-water visual ship hull inspection using a hover-capable underwater vehicle with stereo vision. J Field Robotics 36(3), 531–546 (2019). https://doi.org/10.1002/rob.21841
Kazmi, W., Ridao, P., Ribas, D., Hernandez, E.: Dam wall detection and tracking using a mechanically scanned imaging sonar. In: 2009 IEEE International Conference on Robotics and Automation, pp. 3595–3600 (2009). https://doi.org/10.1109/ROBOT.2009.5152691
Fischler, M.A., Bolles, R.C.: Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981). https://doi.org/10.1145/358669.358692
Karras, G.C., Bechlioulis, C.P., Abdella, H.K., Larkworthy, T., Kyriakopoulos, K., Lane D.: A robust sonar servo control scheme for wall-following using an autonomous underwater vehicle. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3893–3898 (2013). https://doi.org/10.1109/IROS.2013.6696913
Wang, X., Zhang, G., Sun, Y., Wan, L., Cao, J.: Research on autonomous underwater vehicle wall following based on reinforcement learning and multi-sonar weighted round robin mode. Int. J. Adv. Robotic Syst. 17(3) (2020). https://doi.org/10.1177/1729881420925311
Fossen T.: Handbook of Marine Craft Hydrodynamics and Motion Control, pp. 331–387. John Wiley & Sons, Ltd (2011). Chap. 12. https://doi.org/10.1002/9781119994138
Amundsen, H.B., Caharija, W., Pettersen, K.Y.: Autonomous rov inspections of aquaculture net pens using dvl. IEEE J. Oceanic Eng. 47(1), 1–19 (2022). https://doi.org/10.1109/JOE.2021.3105285
Arnesen, B.O., Lekkas, A.M., Schjølberg I.: 3D Path Following and Tracking for an Inspection Class ROV. International Conference on Offshore Mechanics and Arctic Engineering, vol. Volume 7A: Ocean Engineering, pp. 07–06019 (2017). https://doi.org/10.1115/OMAE2017-61170
Galceran, E., Palomeras, N., Carreras, M.: Profile following for inspection of underwater structures. Paladyn, J. Behavioral Robotics 4(4), 211–222 (2013) https://doi.org/10.2478/pjbr-2013-0019
Nguyen, V.S., Trinh, T.H., Tran, M.H.: Hole boundary detection of a surface of 3d point clouds. In: 2015 International Conference on Advanced Computing and Applications (ACOMP), pp. 124–129 (2015). https://doi.org/10.1109/ACOMP.2015.12
Nguyen, V.-S., Bac, A., Daniel, M.: Boundary extraction and simplification of a surface defined by a sparse 3d volume. In: Proceedings of the Third Symposium on Information and Communication Technology. SoICT ’12, pp. 115–124. Association for Computing Machinery, New York, NY, USA (2012). https://doi.org/10.1145/2350716.2350735
Gai, S., Da, F., Zeng, L., Huang, Y.: Research on a hole filling algorithm of a point cloud based on structure from motion. J. Opt. Soc. Am. A 36(2), 39–46 (2019). https://doi.org/10.1364/JOSAA.36.000A39
Fiolka, T., Rouatbi, F., Bender, D.: Automated Detection and Closing of Holes in Aerial Point Clouds Using AN Uas. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 42W6, 101–107 (2017) https://doi.org/10.5194/isprs-archives-XLII-2-W6-101-2017
Cardaillac, A., Ludvigsen, M.: Ruled path planning framework for safe and dynamic navigation. In: OCEANS 2021: San Diego - Porto, pp. 1–7 (2021). https://doi.org/10.23919/OCEANS44145.2021.9705699
Scheiber, M., Cardaillac, A., Brommer, C., Weiss, S., Ludvigsen, M.: Modular multi-sensor fusion for underwater localization for autonomous rov operations. In: OCEANS 2022, Hampton Roads, pp. 1–5 (2022). https://doi.org/10.1109/OCEANS47191.2022.9977298
Cardaillac, A., Ludvigsen, M.: Path following for underwater inspection allowing manoeuvring constraints. In: Petrovic I., Menegatti, E., Marković, I. (eds.) Intelligent Autonomous Systems 17, pp. 867–880. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-22216-0_58
Maurelli, F., Krupiński, S., Xiang, X., Petillot, Y.: Auv localisation: a review of passive and active techniques. Int. J. Intell. Robotics Appl. 6(2), 246–269 (2022). https://doi.org/10.1007/s41315-021-00215-x
Gómez-Espinosa, A., Cuan-Urquizo, E., González-García, J.: Autonomous underwater vehicles: Localization, navigation, and communication for collaborative missions. Appl. Sci. 10, 1256 (2020). https://doi.org/10.3390/app10041256
Brommer, C., Jung, R., Steinbrener, J., Weiss, S.: Mars: A modular and robust sensor-fusion framework. IEEE Robotics Automation Lett. 6(2), 359–366 (2021). https://doi.org/10.1109/LRA.2020.3043195
LaValle, S.M.: Rapidly-exploring random trees: a new tool for path planning. Technical report, Computer Science Dept, Iowa State University (1998)
Dijkstra, E.W.: A note on two problems in connexion with graphs. Numer. Math. 1(1), 269–271 (1959). https://doi.org/10.1007/BF01386390
Koenig, S., Likhachev, M.: Fast replanning for navigation in unknown terrain. IEEE Trans. Robotics 21(3), 354–363 (2005). https://doi.org/10.1109/TRO.2004.838026
Nguyen, V., Martinelli, A., Tomatis, N., Siegwart, R.: A comparison of line extraction algorithms using 2d laser rangefinder for indoor mobile robotics. In: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1929–1934 (2005). https://doi.org/10.1109/IROS.2005.1545234
Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Machine Intell. PAMI-8(6), 679–698 (1986) https://doi.org/10.1109/TPAMI.1986.4767851
Davies, E.R.: Chapter 1 - the dramatically changing face of computer vision. In: Davies E.R., Turk M.A. (eds.) Advanced Methods and Deep Learning in Computer Vision. Computer Vision and Pattern Recognition, pp. 1–91. Academic Press, UK (2022). https://doi.org/10.1016/B978-0-12-822109-9.00010-2
Breivik, M., Fossen, T.I.: Principles of guidance-based path following in 2d and 3d. In: Proceedings of the 44th IEEE Conference on Decision and Control, pp. 627–634 (2005). https://doi.org/10.1109/CDC.2005.1582226
Breivik, M., Fossen, T.I.: Guidance laws for autonomous underwater vehicles. In: Inzartsev A.V. (ed.) Underwater Vehicles, pp. 51–76. IntechOpen, Rijeka (2009). Chap. 4. https://doi.org/10.5772/6696
Lekkas, A.M., Fossen, T.I.: A time-varying lookahead distance guidance law for path following. IFAC Proceedings Volumes, 9th IFAC Conference on Manoeuvring and Control of Marine Craft. 45(27), 398–403 (2012) https://doi.org/10.3182/20120919-3-IT-2046.00068
Lekkas, A.M., Fossen, T.I.: Integral los path following for curved paths based on a monotone cubic hermite spline parametrization. IEEE Trans. Control Syst. Technol. 22(6), 2287–2301 (2014). https://doi.org/10.1109/TCST.2014.2306774
Skjetne, R.: The maneuvering problem. PhD thesis, Norwegian University of Science and Technology (NTNU), Trondheim, Norway (2005)
Hauser, J., Hindman, R.: Maneuver regulation from trajectory tracking: Feedback linearizable systems*. IFAC Proceedings Volumes, 3rd IFAC Symposium on Nonlinear Control Systems Design 1995, Tahoe City, CA, USA, 25-28 June 1995. 28(14), 595–600 (1995). https://doi.org/10.1016/S1474-6670(17)46893-5
Sørensen, M.E.N., Breivik, M., Skjetne, R.: Comparing combinations of linear and nonlinear feedback terms for ship motion control. IEEE Access 8, 193813–193826 (2020). https://doi.org/10.1109/ACCESS.2020.3033409
Breivik, M., Strand, J., Fossen, T.: Guided dynamic positioning for fully actuated marine surface vessels. In: 7th IFAC Conference on Manoeuvring and Control of Marine Craft, pp. 1–6 (2006)
Dia, R., Mottin, J., Rakotovao, T., Puschini, D., Lesecq, S.: Evaluation of occupancy grid resolution through a novel approach for inverse sensor modeling. IFAC-PapersOnLine, 20th IFAC World Congress. 50(1), 13841–13847 (2017) https://doi.org/10.1016/j.ifacol.2017.08.2225
Cheng, C., Wang, C., Yang, D., Liu, W., Zhang, F.: Underwater localization and mapping based on multi-beam forward looking sonar. Frontiers in Neurorobotics 15 (2022). https://doi.org/10.3389/fnbot.2021.801956
Paindaveine, D.: A canonical definition of shape. Stat. & Probability Lett. 78(14), 2240–2247 (2008). https://doi.org/10.1016/j.spl.2008.01.094
Tyler, D.E.: A Distribution-Free \(M\)-Estimator of Multivariate Scatter. Ann. Stat. 15(1), 234–251 (1987). https://doi.org/10.1214/aos/1176350263
Dümbgen, L.: On tyler’s m-functional of scatter in high dimension. Ann. Institute Stat. Math. 50(3), 471–491 (1998). https://doi.org/10.1023/A:1003573311481
Cardaillac, A., Amundsen, H.B., Kelasidi, E., Ludvigsen, M.: Application of maneuvering based control for autonomous inspection of aquaculture net pens. In: 2023 8th Asia-Pacific Conference on Intelligent Robot Systems (ACIRS), pp. 44–51 (2023). https://doi.org/10.1109/ACIRS58671.2023.10239708
Kim, A., Eustice, R.M.: Real-time visual slam for autonomous underwater hull inspection using visual saliency. IEEE Trans. Robotics 29(3), 719–733 (2013). https://doi.org/10.1109/TRO.2012.2235699
Cardaillac, A., Ludvigsen, M.: A communication interface for multilayer cloud computing architecture for low cost underwater vehicles*. IFAC-PapersOnLine, 11th IFAC Symposium on Intelligent Autonomous Vehicles IAV 2022. 55(14), 77–82 (2022). https://doi.org/10.1016/j.ifacol.2022.07.586
Waszak, M., Cardaillac, A., Elvesæter, B., Rødølen, F., Ludvigsen, M.: Semantic segmentation in underwater ship inspections: Benchmark and data set. IEEE J. Oceanic Eng. 1–12 (2022) https://doi.org/10.1109/JOE.2022.3219129
Cardaillac, A., Ludvigsen, M.: Camera-sonar combination for improved underwater localization and mapping. IEEE Access 11, 123070–123079 (2023). https://doi.org/10.1109/ACCESS.2023.3329834
Lu, W., Cheng, K., Hu, M.: Reinforcement learning for autonomous underwater vehicles via data-informed domain randomization. Appl. Sci. 13(3) (2023). https://doi.org/10.3390/app13031723
Hadi, B., Khosravi, A., Sarhadi, P.: Deep reinforcement learning for adaptive path planning and control of an autonomous underwater vehicle. Appl. Ocean Res. 129, 103326 (2022). https://doi.org/10.1016/j.apor.2022.103326
Acknowledgements
The authors would like to thank the collaborators within the BugWright2 projects as well as Borja Serra from Blueye Robotics for their continuous technical support within this project.
Funding
Open access funding provided by NTNU Norwegian University of Science and Technology (incl St. Olavs Hospital - Trondheim University Hospital). This work was supported by the BugWright2 EU H2020-Project [Grant No. 871260]; and by the Research Council of Norway through the Centre of Excellence NTNU AMOS [Grant No. 223254].
Author information
Authors and Affiliations
Contributions
The first author (A. Cardaillac) performed the research and experiments. The second and third authors contributed to the study conception and design (R. Skjetne and M. Ludvigsen).
Corresponding author
Ethics declarations
Conflicts of interest
The authors have no conflict of interest to declare.
Ethics approval
Not applicable.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Cardaillac, A., Skjetne, R. & Ludvigsen, M. ROV-Based Autonomous Maneuvering for Ship Hull Inspection with Coverage Monitoring. J Intell Robot Syst 110, 59 (2024). https://doi.org/10.1007/s10846-024-02095-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-024-02095-2