GROUP VIDEO NAVIGATION OF HETEROGENEOUS ROBOTS
Abstract
This paper addresses the pressing challenges of collaborative autonomous video navigation of unmanned aerial vehicles and ground robots in urban environments, including dense urban development and buildings, as well as in rugged terrain, including mountainous and wooded areas, where, as in urban environments, the use of traditional remote control and navigation tools may be limited by the presence of shielded areas. It is proposed to solve group navigation problems using data from an onboard vision system during operational reconnaissance of the work area by an unmanned aerial vehicle. The results ensure autonomous movement and flight of both individual heterogeneous robotic systems and in a group. The navigation algorithms are based on the methods and algorithms for processing data from the onboard vision system, consisting of a complex of mutually adjusted lidar, television camera and thermal imager, which form the geometry of the surrounding space in the form of a point cloud with the distribution of color and temperature fields on it, allowing for the effective solution of the SLAM problem (determination of the current coordinates of the control object with the formation of a geometric model of the external environment) and the classification of the working area according to the criteria of geometric and support cross-country ability, which ensures autonomous flight and movement of robots for air and ground use in urbanized environments and on rough terrain. It is proposed to use an information and navigation field, represented as a visibility graph, to organize the autonomous operation of aerial and ground robots, including in a group, and a set of reference images, allowing for the correct execution of planned trajectories, taking into account errors in the video navigation task. This information and navigation field allows for the compact presentation of information necessary and sufficient for the autonomous operation of unmanned aerial vehicles and ground robots and facilitates its exchange between group members. The results of experimental studies in real-world conditions of urbanized environments and rugged terrain are presented, confirming the effectiveness of the proposed methods, algorithms, and corresponding software and hardware
References
1. Noskov V.P., Barichev Yu.S., Goydin O.P., Kur'yanov A.N. Razrabotka i issledovanie sredstv video-dal'nometricheskoy navigatsii robotov vozdushnogo i nazemnogo primeneniya [Development and study of video-range-metric navigation tools for air and ground robots], Izvestiya YuFU. Tekhnicheskie nauki [Izvestiya SFedU. Engineering Sciences], 2025, No. 2, pp. 96-108.
2. Noskov V.P. Videonavigatsiya avtonomnykh robotov: ucheb. posobie [Video navigation of autonomous robots: a tutorial]. Moscow: Izd-vo MGTU im. N.E. Baumana, 2025, 230 p.
3. Noskov V.P., Rubtsov I.V., Romanov A.Yu. Formirovanie ob"edinennoy modeli vneshney sredy na os-nove informatsii videokamery i dal'nomera [Formation of a combined model of the external environment based on the information from a video camera and a rangefinder], Mekhatronika, avtomatizatsiya, uprav-lenie [Mechatronics, Automation, Control], 2007, No. 8, pp. 2-5.
4. Noskov V.P., Kiselev I.O. Trekhmernyy variant metoda Khafa v rekonstruktsii vneshney sredy i navi-gatsii [Three-dimensional version of the Hough method in the reconstruction of the external environment and navigation], Mekhatronika, avtomatizatsiya, upravlenie [Mechatronics, Automation, Control], 2018, No. 8, pp. 552-560.
5. Noskov V.P., Kiselev I.O. Vydelenie ploskikh ob"ektov v lineyno strukturirovannykh 3D-izobrazheniyakh [Selection of flat objects in linearly structured 3D images], Robototekhnika i tekhnich-eskaya kibernetika [Robotics and technical cybernetics], 2018, No. 2 (19), pp. 31-38.
6. Konouchine A., Gaganov V., Veznevets V. AMLESAC: A new maximum likelihood robust estimator, Proc. Graphicon, 2005, Vol. 5, pp. 93-100.
7. Klasing K. et al. Comparison of surface normal estimation methods for range sensing applications, Ro-botics and Automation, 2009. ICRA'09. IEEE International Conference on. IEEE, 2009, pp. 3206-3211.
8. Holz D. et al. Real-time plane segmentation using RGB-D cameras, Robot Soccer World Cup. Springer, Berlin, Heidelberg, 2011, pp. 306-317.
9. Kaz'min V.N., Noskov V.P. Vydelenie geometricheskikh i semanticheskikh ob"ektov v dal'nometrich-eskikh izobrazheniyakh dlya navigatsii robotov i rekonstruktsii vneshney sredy [Extraction of geometric and semantic objects in ranging images for robot navigation and environmental reconstruction], Izvestiya YuFU. Tekhnicheskie nauki [Izvestiya SFedU. Engineering Sciences], 2015, No. 10 (171), pp. 71-83.
10. Noskov V.P., Kiselev I.O. Ispol'zovanie tekstury lineynykh ob"ektov dlya postroeniya modeli vneshney sredy i navigatsii [Using the texture of linear objects to construct a model of the external environment and navigation], Mekhatronika, avtomatizatsiya, upravlenie [Mechatronics, Automation, Control], 2019, No. 8, pp. 490-497.
11. Noskov V.P., Gubernatorov D.V. Ekstremal'naya navigatsiya po 3D-izobrazheniyam v mobil'noy roboto-tekhnike [Extreme navigation based on 3D images in mobile robotics], Mekhatronika, avtomatizatsiya, upravlenie [Mechatronics, Automation, Control], 2021, Vol. 22, No. 11, pp. 594-600.
12. Zhang, Zhengyou. Iterative point matching for registration of free-form curves and surfaces, Interna-tional Journal of Computer Vision, 1994, 13 (12), pp. 119-152.
13. Nüchter A., Lingemann K., Hertzberg J., Surmann H. 6D SLAM - 3D Mapping Outdoor Environ-ments, Journal of Field Robotics, September 2007, Vol. 24, No. 8-9, pp. 699-722.
14. Segal, A., Haehnel, D., Thrun, S. Generalized-ICP, Proc. of Robotics: Science and Systems, RSS, 2009.
15. Pomerleau F., Colas F., Siegwart R., Magnenat S. Comparing ICP Variants on Real-World Data Sets, Autonomous Robots, April 2013, Vol. 34, No. 3, pp. 133-148.
16. Noskov V.P., Kur'yanov A.N. Ispol'zovanie kompleksirovannykh deskriptorov v reshenii SLAM-zadachi [Using complexed descriptors in solving the SLAM problem], Izvestiya YuFU. Tekhnicheskie nauki [Izvestiya SFedU. Engineering Sciences], 2022, No. 1, pp. 268-278.
17. Buyvolov G.A., Noskov V.P., Rurenko A.A., Raspopin A.N. Apparatno-algoritmicheskie sredstva formi-rovaniya modeli problemnoy sredy v usloviyakh peresechennoy mestnosti [Hardware and Algorithmic Means for Forming a Model of a Problem Environment in Rough Terrain Conditions], Sb. nauchnykh trudov «Upravlenie dvizheniem i tekhnicheskoe zrenie avtonomnykh transportnykh robotov» [Collection of scientific papers "Motion Control and Technical Vision of Autonomous Transport Robots"]. Mos-cow: IFTP, 1989, pp. 61-69.
18. Vazaev A.V., Noskov V.P., Rubtsov I.V., Tsarichenko S.G. Raspoznavanie ob"ektov i tipov opornoy poverkhnosti po dannym kompleksirovannoy sistemy tekhnicheskogo zreniya [Recognition of objects and types of supporting surfaces based on data from an integrated machine vision system], Izvestiya YuFU. Tekhnicheskie nauki [Izvestiya SFedU. Engineering Sciences], 2016, No. 2 (175), pp. 127-139.
19. Vazaev A.V., Mashkov K.Yu., Noskov V.P., Rubtsov I.V. Klassifikatsiya zony manevrirovaniya RTK na osnove taktil'noy i zritel'noy informatsii [Classification of the maneuvering zone of the robotic complex based on tactile and visual information], Izvestiya YuFU. Tekhnicheskie nauki [Izvestiya SFedU. Engi-neering Sciences], 2021, No.№ 1, pp. 6-19.
20. Dijkstra E.W. A note on two problems in connexion with graf, Numerische Mathematik, 1959, Vol. 1, Iss. 1, pp. 269-271. ISSN 0029-590X. 0945-3245. doi 10.1007/BF01386390
