Abstract
Aquatic macrophyte is a generic denomination for macro-algae with active photosynthetic parts that remain totally or partially submerged in fresh or salty water, in rivers and lakes. Currently, algae monitoring is carried out manually by collecting samples to send for laboratory analysis. In most cases, harmful algal blooms are already widespread when the results are disclosed. This paper proposes the application of a team of heterogeneous Unmanned Aerial Vehicles (UAVs) that cooperate to increase the system’s overall observation range and reduce the reaction time. Leader UAV, featured with a deep-learning-based vision system, covers a pre-determined region and determines high-interest inspection areas in real-time. Through a multi-robot Informative Path Planning (MIPP) approach, the leader UAV coordinates a team of customized quadcopter (named ART2) to reach points of interest, managing their route dynamically. ART2s are able to land on water, and collect and test samples in situ by applying phosphorescence sensors. While path planning, task assignment, and route management are centralized operations, each UAV is conducted by a decentralized trajectory tracking control. Simulations performed in a realistic environment implemented on the Unity platform and experimental proof of concepts demonstrated the reliability of the proposed approach. The presented multi-UAV framework with heterogeneous agents also enables the reconfiguration and expansion of specific objectives, in addition to minimizing the task completion time by executing different processes in parallel. This preventive monitoring enables a plague control action in advance, solving it faster, cheaper, and more effectively.
Article PDF
Similar content being viewed by others
References
Kim, J., Eggleton, CD., SA Wilkerson, SA Gadsden. Cooperative sensor-based selective graph exploration strategy for a team of quadrotors. Journal of Intelligent and Robotic Systems. 103(24),1–14, 2021. https://doi.org/10.1007/s10846-021-01485-0
Burdziakowski, P.: Increasing the geometrical and interpretation quality of unmanned aerial vehicle photogrammetry products using super-resolution algorithms. Remote Sensing. 12(5), 28–33 (2020). https://doi.org/10.3390/rs12050810
Yu, Z., Sun F., Lu, X., Song, Y. Overview of research on 3D path planning methods for rotor UAV. In International Conference on Electronics, Circuits and Information Engineering (ECIE), pages 368–371. IEEE, 2021. https://doi.org/10.1109/ECIE52353.2021.00081
Dutta, A., Ghosh, A., Kreidl, O.P. Multi-robot informative path planning with continuous connectivity constraints. In IEEE International Conference on Robotics and Automation (ICRA), pages 3245–3251 (2019). IEEE. ISBN 9781538660263.https://doi.org/10.1109/ICRA.2019.8794090
Stache, F., Westheider, J., Magistri, F., Popović, M., Stachniss, C. Adaptive path planning for UAV-based multi-resolution semantic segmentation. In European Conference on Mobile Robots (ECMR), pages 1–6 (2021). IEEE. https://doi.org/10.1109/ECMR50962.2021.9568788
Dutta, A., ,Bhattacharya, A., Kreidl, O.P., Ghosh, A., Dasgupta, P. Multi-robot informative path planning in unknown environments through continuous region partitioning. International Journal of Advanced Robotic Systems. 17(6) (2020). https://doi.org/10.1177/1729881420970461
Matoui, F., Boussaid, B., Metoui, B., Abdelkrim, M.N.: Contribution to the path planning of a multi-robot system: centralized architecture. Intel. Serv. Robot. 13(1), 147–158 (2020). https://doi.org/10.1007/s11370-019-00302-w
Pugliese, P. Decentralized control for multi-robot systems with event-triggered communications. Master’s thesis, Department of Mechatronics Engineering, Politecnico di Torino (2020)
Viseras, A., Shutin, D., Merino, L. Robotic active information gathering for spatial field reconstruction with rapidly-exploring random trees and online learning of gaussian processes. Sensors (Switzerland). 19(5) (2019). ISSN 14248220. https://doi.org/10.3390/s19051016
Corah, M., Michael, N. Efficient online multi-robot exploration via distributed sequential greedy assignment. Robotics: Science and Systems. 13 (2017). ISSN 2330765X. https://doi.org/10.15607/rss.2017.xiii.070
Yang, Y., Juntao, L., Lingling, P.: Multi-robot path planning based on a deep reinforcement learning DQN algorithm. CAAI Transactions on Intelligence Technology. 5(3), 177–183 (2020). https://doi.org/10.1049/trit.2020.0024
Roghair, J., Niaraki, A., Ko, K., Jannesari, A. A vision based deep reinforcement learning algorithm for UAV obstacle avoidance. In: SAI Intelligent Systems Conference (IntelliSys), pages 115–128 (2022). Springer. ISBN 978-3-030-82193-7. https://doi.org/10.1007/978-3-030-82193-7_8
Samantaray, A., Yang, B., Dietz, J.E., Min, B.-C. Algae detection using computer vision and deep learning (2018)
Ghorbani, Z., Behzadan, A.H. Identification and instance segmentation of oil spills using deep neural networks. In: World Congress on Civil, Structural, and Environmental Engineering, pages 140–148 (2020). https://doi.org/10.11159/iceptp20.140
Saheba Bhatnagar, Laurence Gill, and Bidisha Ghosh. Drone image segmentation using machine and deep learning for mapping raised bog vegetation communities. Remote Sensing. 12(16) (2020). https://doi.org/10.3390/RS12162602
Olov Andersson, Mariusz Wzorek, Piotr Rudol, and Patrick Doherty. Model-predictive control with stochastic collision avoidance using bayesian policy optimization. In: IEEE International Conference on Robotics and Automation (ICRA), pages 4597–4604 (2016). IEEE. https://doi.org/10.1109/ICRA.2016.7487661
Aoki, Y., Asano, Y., Honda, A., Motooka, N., Ohtsuka, T. Nonlinear model predictive control of position and attitude in a hexacopter with three failed rotors. In: 6th IFAC Conference on Nonlinear Model Predictive Control (NMPC). 51 228–233 (2018). https://doi.org/10.1016/j.ifacol.2018.11.018
Chen, S.W., Saulnier, K., Atanasov, N.A., Lee, D.D., Kumar, V.R., Pappas, G.J., Morari, M. Approximating explicit model predictive control using constrained neural networks. In: Annual American Control Conference (ACC), pages 1520–1527 (2018). https://doi.org/10.23919/ACC.2018.8431275
Oyama, Y., Matsushita, B., Fukushima, T. Distinguishing surface fletcher2010 blooms and aquatic macrophytes using Landsat/TM and ETM+ shortwave infrared bands. Remote Sensing of Environment. 157, 35–47 (2015). Special Issue: Remote Sensing of Inland Waters. https://doi.org/10.1016/j.rse.2014.04.031
Dodds, W.K., Bouska, W.W., Eitzmann, J.L., Pilger, T.J., Pitts, K.L., Riley, A.J., Schloesser, J.T., Thornbrugh, D.J. Eutrophication of u.s. freshwaters: Analysis of potential economic damages. Environmental Science & Technology. 43(1), 12–19 (2009). https://doi.org/10.1021/es801217q. PMID: 19209578
Codd, G.A.: fletcher2010 toxins, the perception of water quality, and the prioritisation of eutrophication control. Ecol. Eng. 16(1), 51–60 (2000)
Carpenter, S.R., Caraco, N.F., Correll, D.L., Howarth, R.W., Sharpley, A.N., Smith, V.H.: Nonpoint pollution of surface waters with phosphorus and nitrogen. Ecol. Appl. 8(3), 559–568 (1998). https://doi.org/10.2307/2641247
Havens, K.E. Cyanobacteria blooms: effects on aquatic ecosystems. pfletcher harmful algal blooms: state of the science and research needs. pages 733–747 (2008)
Chambers, P.A., Lacoul, P., Murphy, K.J., Thomaz, S.M.: Global diversity of aquatic macrophytes in freshwater. Hydrobiologia 595(1), 9–26 (2008). https://doi.org/10.1007/s10750-007-9154-6
Fleming, J.P., Dibble, E.D.: Ecological mechanisms of invasion success in aquatic macrophytes. Hydrobiologia 746(1), 23–37 (2015). https://doi.org/10.1007/s10750-014-2026-y
Ersoy, Z., Scharfenberger, U., Baho, D.L., Bucak, T., Feldmann, T., Hejzlar, J., Levi, E., Mahdy, A., Nõges, T., Papastergiadou, E., et al.: Impact of nutrients and water level changes on submerged macrophytes along a temperature gradient: A pan-european mesocosm experiment. Glob. Change Biol. 26(12), 6831–6851 (2020). https://doi.org/10.1111/gcb.15338
Dar, S.H., Kumawat, D.M., Singh, N., Wani, K.A.: Sewage treatment potential of water hyacinth (eichhornia crassipes). Res. J. Environ. Sci. 5(4), 377–385 (2011). https://doi.org/10.3923/rjes.2011.377.385
Sood, A., Uniyal, P., Prasanna, R., Ahluwalia, A. Phytoremediation potential of aquatic macrophyte, Azolla. Ambio. 41, 122–37, 03 (2012). https://doi.org/10.1007/s13280-011-0159-z. ISSN 1654-7209
Frankovich, T.A., Gaiser, E.E., Zieman, J.C., Wachnicka, A.H.: Spatial and temporal distributions of epiphytic diatoms growing on Thalassia testudinum Banks ex König: relationships to water quality. Hydrobiologia 569(1), 259–271 (2006). https://doi.org/10.1007/s10750-006-0136-x
Malthus, T.J. Bio-optical modeling and remote sensing of aquatic macrophytes. In Mishra, D.R., Ogashawara, I., Gitelson, A.A. (eds.) Bio-optical Modeling and Remote Sensing of Inland Waters. chapter 9, pages 263–308 (2017). Elsevier. https://doi.org/10.1016/B978-0-12-804644-9.00009-4. ISBN 978-0-12-804644-9
Conti, L. Algas na barragem UHE Montante Tibagi (2020). https://youtu.be/-i_-uHTBWOU
Sondergaard, M., Johansson, L.S., Lauridsen, T.L., Jorgensen, T.B., Liboriussen, L., Jeppesen, E.: Submerged macrophytes as indicators of the ecological quality of lakes. Freshw. Biol. 55(4), 893–908 (2010). https://doi.org/10.1111/j.1365-2427.2009.02331.x
Brian Ginn. Distribution and limnological drivers of submerged aquatic plant communities in Lake Simcoe (Ontario, Canada): Utility of macrophytes as bioindicators of lake trophic status. Lancet. 37, 83–89, 12 (2011). https://doi.org/10.1016/j.jglr.2011.03.015
Daniel Canfield, Jerome Shireman, Douglas Colle, William Haller, Curtis II, and Michael Maceina. Prediction of chlorophyll a concentrations in Florida lakes: Importance of aquatic macrophytes. Canadian Journal of Fisheries and Aquatic Sciences. 41, 497–501, 04 (2011). https://doi.org/10.1139/f84-059
Son, D., Cho, H., Lee, E.J. Determining factors for the occurrence and richness of submerged macrophytes in major korean rivers. Aquatic Botany. 150, 82–88, 07 (2018). https://doi.org/10.1016/j.aquabot.2018.07.003
Kislik, C., Genzoli, L., Lyons, A., Kelly, M. Application of UAV imagery to detect and quantify submerged filamentous algae and rooted macrophytes in a non-wadeable river. Remote Sensing. 12, 3332, 10 (2020). https://doi.org/10.3390/rs12203332
Luo, S., Kim, J., Min, B.-C.: Asymptotic boundary shrink control with multirobot systems. IEEE Transactions on Systems, Man, and Cybernetics: Systems. 52(1), 591–605 (2022). https://doi.org/10.1109/TSMC.2020.3003824
Penmetcha, M., Luo, S., Samantaray, A., Dietz, J.E, Yang, B., Min, B.-C. Computer vision-based algae removal planner for multi-robot teams. In: 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). pages 1575–1581 (2019). https://doi.org/10.1109/SMC.2019.8913967
Gong, X., Ma, C., Sun, B., Zhang, J. An efficient self-organized detection system for algae. Sensors. 23(3) (2023). https://www.mdpi.com/1424-8220/23/3/1609. ISSN 1424-8220
Wu, D., Li, R., Liu, J., Khan, N.: Monitoring algal blooms in small lakes using drones: a case study in southern illinois. Journal of Contemporary Water Research & Education. 177(1), 83–93 (2023)
Li, Jing, Xiong, Yonghua, She, Jinhua, Min, Wu.: A path planning method for sweep coverage with multiple UAVs. IEEE Internet Things J. 7(9), 8967–8978 (2020). https://doi.org/10.1109/JIOT.2020.2999083
Pereira, J.L. Hover and wind-tunnel testing of shrouded rotors for improved micro air vehicle design. Master’s thesis, University of Maryland, College Park (2008). https://drum.lib.umd.edu/handle/1903/8752
Amarante, O.A.C., Brower, M., Zack, J., Sá, A.L.: Atlas do potencial eólico brasileiro. Technical report, Ministério de Minas e Energia (2001). (In Portuguese)
Bencatel, R., Souza, J., Faied, M., Girard, A. Shear wind estimation. In AIAA Guidance, Navigation and Control Conference. (2011). https://doi.org/10.2514/6.2011-6224
Troen, I., Petersen, E.L.: European wind atlas. Technical report, Riso National Laboratory, Roskilde (1989)
Baca, T., Petrlik, M., Matous Vrba, Vojtech Spurny, Robert Penicka, Daniel Hert, and Martin Saska. The MRS UAV system: Pushing the frontiers of reproducible research, real-world deployment, and education with autonomous unmanned aerial vehicles. Journal of Intelligent and Robotic Systems: Theory and Applications. 102(1), (2021). https://doi.org/10.1007/s10846-021-01383-5
Lee, T., Leok, M., McClamroch, N.H. Geometric tracking control of a quadrotor uav on se (3). In: 49th IEEE Conference on Decision and Control (CDC), pages 5420–5425 (2010). IEEE. https://doi.org/10.1109/CDC.2010.5717652
Petrlík, M., Báča, T., Heřt, D., Vrba, M., Krajník, T., Saska, M.: A robust UAV system for operations in a constrained environment. IEEE Robotics and Automation Letters. 5(2), 2169–2176 (2020). https://doi.org/10.1109/LRA.2020.2970980
Mehta, B.R., Reddy, Y.J. Advanced process control systems. Industrial Process Automation Systems 547–557 (2015). https://doi.org/10.1016/B978-0-12-800939-0.00019-X
Ziegler, J.G., Nichols, N.B.: Optimum settings for automatic controllers. J. Dyn. Syst. Meas. Contr. 115(2B), 220–222 (1993). https://doi.org/10.1115/1.2899060
Benevides, J.R.S., Inoue, R.S., Paiva, M.A.D., Terra, M.H. ROS-based robust and recursive optimal control of commercial quadrotors. In IEEE International Conference on Automation Science and Engineering (ICASCE). pages 998–1003 (2019). IEEE. https://doi.org/10.1109/COASE.2019.8843004. ISBN 9781728103556
Neil Flood, Fiona Watson, and Lisa Collett. Using a U-net convolutional neural network to map woody vegetation extent from high resolution satellite imagery across Queensland, Australia. International Journal of Applied Earth Observation and Geoinformation. 82, 10 (2019). https://doi.org/10.1016/j.jag.2019.101897
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention (MICCAI), pages 234–241 (2015). Springer. https://doi.org/10.1007/978-3-319-24574-4_28
Sandler, M., Howard, A.G., Zhu, M., Zhmoginov, A., Chen, L.-C. Inverted residuals and linear bottlenecks: mobile networks for classification, detection and segmentation (2018). arXiv:1801.04381
The GIMP Development Team. Gimp (2019). https://www.gimp.org
Haas, J.K.: A history of the Unity game engine. Technical report, Worcester Polytechnic Institute (2014)
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pages 248–255 (2009). IEEE. https://doi.org/10.1109/CVPR.2009.5206848
Amina Khan, Sumeet Gupta, and Sachin Kumar Gupta. Cooperative control between multi-UAVs for maximum coverage in disaster management: review and proposed model. In: International Conference on Computing and Information Technology (ICCIT). 2, pages 271–277 (2022). IEEE https://doi.org/10.1109/ICCIT52419.2022.9711627
Lakshmanan, A.K., Mohan, R.E., Ramalingam, B., Le, A.V., Veerajagadeshwar, P., Tiwari, K., Ilyas, M.: Complete coverage path planning using reinforcement learning for tetromino based cleaning and maintenance robot. Autom. Constr. 112, 103078 (2020). https://doi.org/10.1016/j.autcon.2020.103078
Shen, Z., Wilson, J.P., Gupta, S. \(\epsilon ^{\star }+\): An online coverage path planning algorithm for energy-constrained autonomous vehicles. In Global Oceans 2020: Singapore–US Gulf Coast. pages 1–6 (2020). IEEE. https://doi.org/10.1109/IEEECONF38699.2020.9389353
Hari, S.K.K., Rathinam, S., Darbha, S., Kalyanam, K., Manyam, S.G., Casbeer, D.: Optimal UAV route planning for persistent monitoring missions. IEEE Trans. Rob. 37(2), 550–566 (2020). https://doi.org/10.1109/TRO.2020.3032171
Wang, X., Wang, S., Wang, L., Zheng, H., Hao, J., He, R., Sun, Z. An effective iterated greedy algorithm for online route planning problem. In: IEEE Congress on Evolutionary Computation (CEC), pages 1–8 (2020). IEEE. https://doi.org/10.1109/CEC48606.2020.9185864
Acknowledgements
The authors would like to thank the Office of Extension Activities at the Federal University of São Carlos (ProEx/UFSCar-3112.007651/2021-00) for their support, Acqua Nativa company for providing the chlorophyll probe and IADrones for the footage of the experimental tests. The authors would like to acknowledge the support of PhD. Débora Miloni from Embrapa Instrumentation, the agronomist engineer Raquel Stucchi Boschi from UFSCar - General Secretariat for Environmental Management and Sustainability (SGAS) - on the search for experiments location and field campaign, also the geographer and water quality specialist, Prof. Peter Zeilhofer, for his support on the process of image classification and statistical analysis, as well as the Research Group in Geotechnology - GEOTEC [dgp.cnpq.br/dgp/espelhogrupo/3518977307518871] on behalf of Prof. Paulo Henrique Corrêa de Morais, Prof. Roberto Nunes Vianconi Souto and Prof. Robson Rogério Dutra Pereira for providing the images dataset. Finally, the authors would like to thank financial support provided by the Brazilian National Science and Technology Institute for Autonomous Cooperative Systems (INSAC), São Paulo Research Foundation (FAPESP) (grants 2014/50851-0 and #2016/21220-7), National Council for Scientific and Technological Development (CNPq) (grant #465755/2014-3 and #421131/2018-7) and Coordination for the Improvement of Higher Education Personnel (CAPES).
Funding
This work was partially supported by the Brazilian National Science and Technology Institute for Autonomous Cooperative Systems (INSAC), São Paulo Research Foundation (FAPESP) (grants #2014/50851-0 and #2016/21220-7), National Council for Scientific and Technological Development (CNPq) (grants #465755/2014-3 and #421131/2018-7) and Coordination for the Improvement of Higher Education Personnel (CAPES).
Author information
Authors and Affiliations
Contributions
K.V. contributed as a Systems Engineering Manager, coordinating the overall development of the project and advising on the path planning algorithm. K.V. also structured the first draft of the manuscript and closed its final version after the comments and contributions from all authors. T.P. contributed to the support of the systems engineering process, as an advisor of control systems methods, and also to the structuring, writing, and reviewing of the manuscript and auxiliary materials. L.R. developed the path-planning techniques and was responsible for the integration of the simulation environment. L.R. also contributed to the writing of Sections 7, 8, and 9. I.S. contributed as a hardware integration engineer, being responsible for customizing and conducting propulsion system tests on the F450 aircraft. I.S. also conducted propulsion system tests and integrated the AcquaProbe-Cla chlorophyll probe into ART2. Additionally, I.S. is the certified pilot for the flights. K.C. contributed to the development of the sensor fusion method for localization and was involved in the system integration. K.C. was also responsible for the description of the F450 aircraft in the paper. D.S. was responsible for developing the computer vision framework, training neural networks, and preparing the dataset. J.B. developed the control system interface and was responsible for tuning the control system of the F450. In the manuscript, Section 5 includes contributions by J.B.. P.S. was responsible for the design, development, implementation and tuning of control systems, being the main writer of Section 5 in this paper. A.H. was responsible for the conception, development and construction of ART2 aircraft. In the manuscript, Section 4.2 was written by A.H.. K.A. was responsible for the design and development of the simulation environment. P.K. supported the image processing process and the implementation of neural network solutions. P.K. contributed to writing Section 6 in the manuscript. I.A. implemented ART2 in the simulation environment. E.N. supported the development of the simulation environment and was responsible to structure Section 1 in the text. M.S. supported the development of path planning and optimization methods. M.S. wrote the text in Section 8.2 in this manuscript. A.A. supported the computer vision process and the implementation of neural network solutions. L.C. supported the implementation of image processing and neural networks solution in the simulation environment. R.I. assisted in carrying out field experiments and supervising the implementation of control and localization systems for unmanned aerial vehicles. M.T. contributed as an advisor to control systems development. M.B. was responsible for the idea of the problem to be addressed and the initial design solution. M.B. also contributed as an advisor on the development of ART2 aircraft. All authors read and approved the final text of this paper before submission.
Corresponding author
Ethics declarations
Competing interests
The authors have no competing interests to declare that are relevant to the content of this article.
Ethics approval
Not applicable.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Vivaldini, K., Pazelli, T., Rocha, L. et al. Multi-UAV Collaborative System for the Identification of Surface Cyanobacterial Blooms and Aquatic Macrophytes. J Intell Robot Syst 110, 40 (2024). https://doi.org/10.1007/s10846-023-02043-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-023-02043-6