Abstract
Multiple modern robotic applications benefit from centralized cognition and processing schemes. However, modern equipped robotic platforms can output a large amount of data, which may exceed the capabilities of modern wireless communication systems if all data is transmitted without further consideration. This research presents a multi-agent, centralized, and real-time 3D point cloud map merging scheme for ceaselessly connected robotic agents. Centralized architectures enable mission awareness to all agents at all times, making tasks such as search and rescue more effective. The centralized component is placed on an edge server, ensuring low communication latency, while all agents access the server utilizing a fifth-generation (5G) network. In addition, the proposed solution introduces a communication-aware control function that regulates the transmissions of map instances to prevent the creation of significant data congestion and communication latencies as well as address conditions where the robotic agents traverse in limited to no coverage areas. The presented framework is agnostic of the used localization and mapping procedure, while it utilizes the full power of an edge server. Finally, the efficiency of the novel established framework is being experimentally validated based on multiple scenarios.
Article PDF
Similar content being viewed by others
References
Chen, S.W., Nardari, G.V., Lee, E.S., Qu, C., Liu, X., Romero, R.A.F., Kumar, V.: SLOAM: Semantic Lidar Odometry and Mapping for Forest Inventory. IEEE Robot. Autom. Lett. 5(2), 612–619 (2020). https://doi.org/10.1109/LRA.2019.2963823
Thrun, S., Thayer, S., Whittaker, W., Baker, C., Burgard, W., Ferguson, D., Hahnel, D., Montemerlo, D., Morris, A., Omohundro, Z., Reverte, C.: Autonomous exploration and mapping of abandoned mines. IEEE Robot. Autom. Mag. 11(4), 79–91 (2004). https://doi.org/10.1109/MRA.2004.1371614
Jenssen, R., Roverso, D., et al.: Automatic autonomous vision-based power line inspection: A review of current status and the potential role of deep learning. Int J. Electr. Power Energy Syst. 99, 107–120 (2018)
Scaramuzza, D., Achtelik, M.C., Doitsidis, L., Friedrich, F., Kosmatopoulos, E., Martinelli, A., Achtelik, M.W., Chli, M., Chatzichristofis, S., Kneip, L., Gurdan, D., Heng, L., Lee, G.H., Lynen, S., Pollefeys, M., Renzaglia, A., Siegwart, R., Stumpf, J.C., Tanskanen, P., Troiani, C., Weiss, S., Meier, L.: Vision-Controlled Micro Flying Robots: From System Design to Autonomous Navigation and Mapping in GPS-Denied Environments. IEEE Robot. Autom. Mag. 21(3), 26–40 (2014)
Lindqvist, B., Kanellakis, C., Mansouri, S.S., Agha-mohammadi, A.-A., Nikolakopoulos, G.: COMPRA: A COMPact Reactive Autonomy Framework for Subterranean MAV Based Search-And-Rescue Operations. J. Intell. Robot. Syst. 105(3), 49 (2022). https://doi.org/10.1007/s10846-022-01665-6
Agha, A., Otsu, K., Morrell, B., Fan, D.D., Thakker, R., Santamaria-Navarro, A., Kim, S.-K., Bouman, A., Lei, X., Edlund, J.A., Ginting, M.F., Ebadi, K., Anderson, M.O., Pailevanian, T., Terry, E., Wolf, M.T., Tagliabue, A., Vaquero, T.S., Palieri, M., Tepsuporn, S., Chang, Y., Kalantari, A., Chavez, F., Lopez, B.T., Funabiki, N., Miles, G., Touma, T., Buscicchio, A., Tordesillas, J., Alatur, N., Nash, J., Walsh, W., Jung, S., Lee, H., Kanellakis, C., Mayo, J., Harper, S., Kaufmann, M., Dixit, A., Correa, G., Lee, C.-A., Gao, J.L., Merewether, G.B., Maldonado-Contreras, J., Salhotra, G., Silva, M.S.D., Ramtoula, B., Fakoorian, S.A., Hatteland, A., Kim, T., Bartlett, T., Stephens, A., Kim, L., Bergh, C.F., Heiden, E., Lew, T., Cauligi, A., Heywood, T., Kramer, A., Leopold, H., Melikyan, H., Choi, H.-Y., Daftry, S., Toupet, O., Wee, I., Thakur, A., Feras, M., Beltrame, G.A., Nikolakopoulos, G., Shim, D.-Y., Carlone, L., Burdick, J.W.: NeBula: TEAM CoSTAR’s Robotic Autonomy Solution that Won Phase II of DARPA Subterranean Challenge. Field Robot. 2, 1432–1506 (2022)
Saboia, M., Clark, L., Thangavelu, V., Edlund, J.A., Otsu, K., Correa, G.J., Varadharajan, V.S., Santamaria-Navarro, A., Touma, T., Bouman, A., Melikyan, H., Pailevanian, T., Kim, S.-K., Archanian, A., Vaquero, T.S., Beltrame, G., Napp, N., Pessin, G., Agha-mohammadi, A.-a: ACHORD: Communication-Aware Multi-Robot Coordination With Intermittent Connectivity. IEEE Robot. Autom. Lett. 7(4), 10184–10191 (2022). https://doi.org/10.1109/LRA.2022.3193240
Jamshidpey, A., Wahby, M., Heinrich, M.K., Allwright, M., Zhu, W., Dorigo, M.: Centralization vs. decentralization in multi-robot coverage: Ground robots under UAV supervision (2021)
Gielis, J., Shankar, A., Prorok, A.: A Critical Review of Communications in Multi-robot Systems. Curr. Robot. Rep. 3(4), 213–225 (2022). https://doi.org/10.1007/s43154-022-00090-9
Matignon, L., Jeanpierre, L., Mouaddib, A.-I.: Coordinated Multi-Robot Exploration Under Communication Constraints Using Decentralized Markov Decision Processes. Proc. AAAI Conf. Artif. Intell. 26(1), 2017–2023 (2021). https://doi.org/10.1609/aaai.v26i1.8380
Zaki, O., Dunnigan, M., Robu, V., Flynn, D.: Reliability and Safety of Autonomous Systems Based on Semantic Modelling for Self-Certification. Robotics 10(1), (2021). https://doi.org/10.3390/robotics10010010
Damigos, G., Lindgren, T., Sandberg, S., Nikolakopoulos, G.: Performance of Sensor Data Process Offloading on 5G-Enabled UAVs. Sensors 23(2), 864 (2023)
Schmuck, P., Chli, M.: CCM-SLAM: Robust and efficient centralized collaborative monocular simultaneous localization and mapping for robotic teams. J. Field Robot. 36, 763–781 (2019). https://doi.org/10.1002/rob.21854
Zou, D., Tan, P.: Coslam: Collaborative visual slam in dynamic environments. IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 354–366 (2012)
Engel, J., Koltun, V., Cremers, D.: Direct Sparse Odometry. IEEE Trans. Pattern Anal. Mach. Intell. 40(3), 611–625 (2018). https://doi.org/10.1109/TPAMI.2017.2658577
Forster, C., Lynen, S., Kneip, L., Scaramuzza, D.: Collaborative monocular slam with multiple micro aerial vehicles. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3962–3970 (2013). IEEE
Queralta, J.P., Gia, T.N., Tenhunen, H., Westerlund, T.: Collaborative mapping with ioe-based heterogeneous vehicles for enhanced situational awareness. In: 2019 IEEE sensors applications symposium (SAS), pp. 1–6 (2019). IEEE
Hörner, J.: Map-merging for multi-robot system (2016)
Siebert, J.: SIFT Keypoint Descriptors for Range Image Analysis. Annals of the BMVA (2009)
Harris, C., Stephens, M.: A Combined Corner and Edge Detector. In: Proceedings of the 4th alvey vision conference, pp. 147–151 (1988)
Hörner, J.: ROS package for merging 3D point cloud maps. (2018). https://github.com/hrnr/map-merge.git
Drwiega, M.: Incremental 3D Maps Server based on Feature Matching for Multi-robot Systems. In: 2021 25th International conference on methods and models in automation and robotics (MMAR), pp. 342–347 (2021). https://doi.org/10.1109/MMAR49549.2021.9528427
Salti, S., Tombari, F., Di Stefano, L.: SHOT: Unique Signatures of Histograms for Surface and Texture Description. Comput. Vis. Image Underst. 125, (2014). https://doi.org/10.1016/j.cviu.2014.04.011
Stathoulopoulos, N., Koval, A., Agha-mohammadi, A.-a., Nikolakopoulos, G.: FRAME: Fast and Robust Autonomous 3D Point Cloud Map-Merging for Egocentric Multi-Robot Exploration. In: 2023 IEEE international conference on robotics and automation (ICRA), pp. 3483–3489 (2023). https://doi.org/10.1109/ICRA48891.2023.10160771
Dubé, R., Cramariuc, A., Dugas, D., Sommer, H., Dymczyk, M., Nieto, J., Siegwart, R., Cadena, C.: SegMap: Segment-based mapping and localization using data-driven descriptors. Int. J. Robot. Res. 39(2–3), 339–355 (2020). https://doi.org/10.1177/0278364919863090
Stathoulopoulos, N., Koval, A., Nikolakopoulos, G.: 3DEG: Data-Driven Descriptor Extraction for Global re-localization in subterranean environments. Expert Syst. Appl. 237, 121508 (2024). https://doi.org/10.1016/j.eswa.2023.121508
Koide, K., Yokozuka, M., Oishi, S., Banno, A.: Voxelized GICP for Fast and Accurate 3D Point Cloud Registration. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 11054–11059 (2021). https://doi.org/10.1109/ICRA48506.2021.9560835
Segal, A.V., Hähnel, D., Thrun, S.: Generalized-ICP. In: Robotics: Science and Systems (2009)
Shan, T., Englot, B., Meyers, D., Wang, W., Ratti, C., Rus, D.: LIO-SAM: Tightly-coupled LiDAR Inertial Odometry via Smoothing And Mapping. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5135–5142 (2020). IEEE
Palieri, M., Morrell, B., Thakur, A., Ebadi, K., Nash, J., Chatterjee, A., Kanellakis, C., Carlone, L., Guaragnella, C., Agha-mohammadi, A.-a.: LOCUS: A Multi-Sensor Lidar-Centric Solution for High-Precision Odometry and 3D Mapping in Real-Time. IEEE Robot. Autom. Lett. 6(2), 421–428 (2021)
Shan, T., Englot, B., Ratti, C., Daniela, R.: LVI-SAM: Tightly-coupled Lidar-Visual-Inertial Odometry via Smoothing and Mapping. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 5692–5698 (2021). IEEE
Chen, K., Lopez, B.T., Agha-mohammadi, A.-A., Mehta, A.: Direct LiDAR Odometry: Fast Localization With Dense Point Clouds. IEEE Robot. Autom. Lett. 7(2), 2000–2007 (2022). https://doi.org/10.1109/LRA.2022.3142739
Kim, B., Jung, C., Shim, D.H., Agha-mohammadi, A.: Adaptive Keyframe Generation based LiDAR Inertial Odometry for Complex Underground Environments. In: 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 3332–3338 (2023). https://doi.org/10.1109/ICRA48891.2023.10161207
TS 23.501: System architecture for the 5G System (5GS). 3rd Generation Partnership Project (3GPP) (2023). https://www.3gpp.org/
Cloud-Compare: 3D point cloud and mesh processing software Open Source Project. https://www.danielgm.net/cc/
Mogyorósi, F., Revisnyei, P., Pašić, A., Papp, Z., Törös, I., Varga, P., Pašić, A.: Positioning in 5G and 6G networks-A Survey. Sensors 22(13), 4757 (2022)
Chen, X., Läbe, T., Milioto, A., Röhling, T., Vysotska, O., Haag, A., Behley, J., Stachniss, C.: OverlapNet: Loop Closing for LiDAR-based SLAM. In: Proceedings of robotics: science and systems (RSS) (2020)
Stathoulopoulos, N., Pagliari, E., Davoli, L., Nikolakopoulos, G.: Redundant and Loosely Coupled LiDAR-Wi-Fi Integration for Robust Global Localization in Autonomous Mobile Robotics. In: 21st International conference on advanced robotics (ICAR) (2023). https://doi.org/10.48550/arXiv.2310.06384
Ramadan, E., Narayanan, A., Dayalan, U.K., Fezeu, R.A., Qian, F., Zhang, Z.-L.: Case for 5G-aware video streaming applications. In: Proceedings of the 1st workshop on 5g measurements, modeling, and use cases, pp. 27–34 (2021)
Brunello, D., Johansson S, I., Ozger, M., Cavdar, C.: Low Latency Low Loss Scalable Throughput in 5G Networks. In: 2021 IEEE 93rd vehicular technology conference (VTC2021-Spring), pp. 1–7 (2021). https://doi.org/10.1109/VTC2021-Spring51267.2021.9448764
Tzes, A., Nikolakopoulos, G., Koutroulis, I.: Development and experimental verification of a mobile client-centric networked controlled system. Eur. J. Control 11(3), 229–241 (2005)
Thangavelautham, J., Chandra, A., Jensen, E.: Autonomous robot teams for lunar mining base construction and operation. In: 2020 IEEE Aerospace Conference, pp. 1–16 (2020). IEEE
Kubernetes.io: Kubernetes Documentation - General Introduction. https://kubernetes.io/ Accessed 3-Nov-2023
Kubernetes: Kubernetes Documentation. https://www.3gpp.org/technologies/scheduling Accessed 3-Nov-2023
Xia, C., Zhang, Y., Wang, L., Coleman, S., Liu, Y.: Microservice-based cloud robotics system for intelligent space. Robot. Auton. Syst. 110, 139–150 (2018)
Seisa, A.S., Damigos, G., Satpute, S.G., Koval, A., Nikolakopoulos, G.: Edge Computing Architectures for Enabling the Realisation of the Next Generation Robotic Systems. In: 2022 30th Mediterranean conference on control and automation (MED), pp. 487–493 (2022). https://doi.org/10.1109/MED54222.2022.9837289
Funding
Open access funding provided by Lulea University of Technology. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 953454.
Author information
Authors and Affiliations
Contributions
Gerasimos Damigos and Nikolaos Stathoulopoulos: Development, implementation, system integration and field work, relating to all presented submodules and developments, main manuscript contributors. Tore Lindgren, Anton Koval and George Nikolakopoulos: Advisory, manuscript contributions and supervision. All authors have read and approved the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors have no conflicts of interest with any related parties.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Damigos, G., Stathoulopoulos, N., Koval, A. et al. Communication-Aware Control of Large Data Transmissions via Centralized Cognition and 5G Networks for Multi-Robot Map merging. J Intell Robot Syst 110, 22 (2024). https://doi.org/10.1007/s10846-023-02045-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-023-02045-4