MACHINE LEARNING MODEL OF SWARM EVASION FROM THE INFLUENCE OF ANTAGONISTIC ENVIRONMENT
Abstract
One of the priority areas of group control theory for the near future is swarm control of groups of small unmanned aerial vehicles - micro-, mini- and nano-classes, performing a collective task under enemy influence. Here, two antagonistic strategies collide - minimization of losses from the point of view of the attacking swarm and maximization of such losses from the point of view of the defense system. Research objective: development of an approach to solving a practical problem - penetration of a swarm of unmanned aerial vehicles into an object protected by a defense system. The objectives of the study were to analyze the characteristics of the factors influencing the processes of detection, tracking, recognition of swarm intentions by the defense system and the development of a machine learning model for creating spatio-temporal formations that minimize the number of swarm elements affected by the defense system. The main parameters of the defense system are the detection range and duration of swarm recognition, the time to make a decision on the actions of the swarm, the size of the zone of destruction of defense means. The method of machine learning on convolutional neural networks with reinforcement was chosen as the research method. The counteraction effect against the defense system is created due to the swarm's dynamics; it can actively maneuver, creating spatio-temporal maneuvers during the mission. To simulate the "Swarm vs. Defense System" situation, a swarm agent (a neural network with a transformer architecture that initiates swarm formations) and a defense system agent are introduced that recognizes the swarm and attacks it, creating a zone of destruction in the conventional center of mass of the swarm. The swarm is guided by a stochastic rule, asking the defense system (environment) to react to its maneuver. The environment responds by attacking the swarm, creating a damaging factor at the point where the swarm or the main part of the swarm is expected to be. The reward of the swarm strategy is the number of undestroyed objects under the conditions of constraints; for the defense system, this "reward" acts as a "punishment". An interesting phenomenon was established in the process of machine learning: each swarm element, remaining within a given space and implementing the biological principles of swarm control without a Leader, independently evades the area of destruction, which together creates a random spatio-temporal formation for defense means with minimal losses of swarm elements. Thus, using the method of machine learning with reinforcement, a model was created that allows varying the behavior of the swarm and synthesizing spatio-temporal formations that complicate detection, tracking, recognition of intentions and decision-making on the impact of the defense system on a swarm of attacking small unmanned aerial vehicles, as well as significantly reducing their losses.
References
1. Радченко А. Барражирующие боеприпасы и способы борьбы с ними // Армейский сборник.
– 2023. – № 9.
2. Афонин И.Е., Макаренко С.И., Петров С.В., Привалов А.А. Анализ опыта боевого применения
групп беспилотных летательных аппаратов для поражения зенитно-ракетных комплексов сис-
темы противовоздушной обороны в военных конфликтах в Сирии, в Ливии и в Нагорном Кара-
бахе // Системы управления, связи и безопасности. – 2020. – № 4. – С. 163-191. – DOI:
10.24411/2410-9916-2020-10406.
3. Городецкий В.И. Поведенческие модели кибер-физических систем и групповое управление:
основные понятия // Известия ЮФУ. Технические науки. – 2019. – № 1 (203). – С. 144-162.
– DOI: 10.23683/2311-3103-2019-1-144-162.
4. Боровик В.С. [и др.]. Коллективы интеллектуальных роботов. Сферы применения. Серия: “Ин-
теллектуальные технические системы” (подсерия: “Когнитивная робототехника”) / под ред.
В.И. Сырямкина. – Томск: STT, 2018. – 140 с.
5. Huang W., Guo H., Liu J. Task Offloading in UAV Swarm-Based Edge Computing: Grouping and
Role Division // in 2021 IEEE Global Communications Conference (GLOBECOM), 7–11 Dec. 2021.
– P. 1-6. – DOI: 10.109/ GLOBE COM46 510. 2021. 96859 20.
6. Zhu A., Lu H., Ma M., Zhou Z., Zeng Z. DELOFF: decentralized learning-based task offloading for
multi-UAVs in U2X assisted heterogeneous networks // Drones. – 2023. – 7 (11), 656. – P. 1-31.
– DOI: 10.3390/drones7110656.
7. Ростопчин В.В. Ударные беспилотные летательные аппараты и противовоздушная оборона – про-
блемы и перспективы противостояния // Беспилотная авиация. – 2019. – С. 47. – URL:
https://www.researchgate.net/publication/331772628_Udarnye_bespilotnye_letatelnye_apparaty_i_
protivovozdusnaa_ oborona_-problemy_i_perspektivy_protivostoania (дата обращения: 20.02.2025).
8. Макаренко С.И., Тимошенко А.В. Анализ средств и способов противодействия беспилотным
летательным аппаратам. Ч. 2. Огневое поражение и физический перехват // Системы управле-
ния, связи и безопасности. – 2020. – № 1. – С. 147-197. – DOI: 10.24411/2410-9916-2020-10106.
9. Pasha J. et al., The drone scheduling problem: a systematic state-of-the-art review // IEEE Trans.
Intell. Transp. Syst. – 2022. – P. 24. – DOI: 10.1109/TITS.2022.3155072.
10. Abdelkader M., Güler S., Jaleel H., Shamma J. S. Aerial swarms: recent applications and challenges //
Current Robotics Reports. – 2021. – No. 2 (3). – P. 309-320. – DOI: 10. 1007/ s43154- 021- 00063-4.
11. Mohsan S.A.H., Khan M.A., Noor F., Ullah I., Alsharif M.H. Towards the unmanned aerial vehicles
(UAVs): a comprehensive review // Drones. – 2022. – No. 6 (6), 147. – P. 27. – DOI:
10.3390/drones6060147.
12. DARPA-серый кардинал американских технологий. Аналитический обзор // Росконгресс, 06.02.2025.
– URL: https://cdnweb.roscongress.org/upload/medialibrary/ffe/DARPA03.pdf?17388400017118014 (дата
обращения: 20.02.2025).
13. Swarms of AI-fueled drones, vehicles track targets in AUKUS tests. – URL:
https://www.yahoo.com/news/swarms-ai-fueled-drones-vehicles-150216013.html (дата обращения:
20.02.2025).
14. Stephen L. Shield AI unveils V-Bat Teams drone swarm tech, with eye to Replicator // Defense News.
– URL: https://www.yahoo.com/news/shield-ai-unveils-v-bat-185742088.html (дата обращения:
20.02.2025).
15. Бойко А. Perdix. – URL: https://robotrends.ru/robopedia/perdix (дата обращения: 20.02.2025).
16. Беспилотные рои готовятся к бою // Военное обозрение от 24.11.2020. – URL:
https://topwar.ru/177344-bespilotnye-roi-gotovjatsja-k-boju.html (дата обращения: 20.02.2025).
17. Макаренко С.И. Противодействие беспилотным летательным аппаратам: монография. – СПб.:
Наукоемкие технологии, 2020. – 204 с.
18. Распознавание воздушных объектов типа «птицы» по траекторным признакам. – URL:
19. Ерёмин Г.В., Гаврилов А.Д., Назарчук И.И. Малоразмерные беспилотники – новая проблема для
ПВО // Армейский вестник. – 2015. – URL: http://armynews. ru/2015/02/malorazmernyebespilotniki-
novaya-problema-dlya-pvo.
20. Верба В.С., Гаврилов К.Ю., Ильчук А.Р., Татарский Б.Г., Филатов А.А. Радиолокация для всех /
под ред. В.С. Вербы. – М.: Техносфера, 2020. – 504 с.
21. Абросимов В.К. Михайлова Е.С. Методы формирования пространственно- временных скопле-
ний объектов в недружественной среде // Автометрия. – 2023. – Т. 29, № 4. – С. 11-19. – DOI:
10.15372/AUT20230402.
22. Junayed Pasha, Zeinab Elmi, Sumit Purkayastha, Amir M. Fathollahi-Fard, Ying-En Ge, Yui-Yip Lau. The
Drone Scheduling Problem: A Systematic State-of-the-Art Review // IEEE Transactions on Intelligent
Transportation Systems. – 2022. – Vol. 23, Issue 9. – P. 14224-14247. – DOI: 10.1109/TITS.2022.3155072.
23. Qi X., Chong J., Zhang Q., Yang Z. Collaborative computation offloading in the multi-UAV fleeted
mobile edge computing network via connected dominating set // IEEE Trans. Veh. Technol. – 2022.
– 71 (10). – P. 10832-10848. – https://doi. org/ 10. 1109/ tvt. 2022. 31885 54.
24. Vaswani A. et al. Attention is all you need // 31st Conference on Neural Information Processing Systems
(NIPS 2017). – 2017. – P. 11.
25. Wu Х., Gao Z., Yuan S., Hu Q., Dang Z. A dynamic task allocation algorithm for heterogeneous UUV
swarms // Sensors. – 2022. – 22 (6), 2122. – P. 18. – DOI: 10.3390/s22062122.
26. Ahmad S., Zhang J., Khan A., Khan U.A., Hayat B. JO-TADP: learning-based cooperative dynamic
resource allocation for MEC–UAV-enabled wireless network // Drones. – 2023. – 7 (5), 303. – P. 35.
– DOI: 10.3390/drones7050303.
27. Bai Y., Zhao H., Zhang X., Chang Z., Jäntti R., Yang K. Toward autonomous multi-UAV wireless
network: a survey of reinforcement learning-based approaches // IEEE Commun. Surv. Tutorials.
– 2023. – 25 (4). – P. 3038-3067. – DOI: 10. 1109/ COMST. 2023. 33233 44.
28. Zhu R., Huang M., Sun K., Hou Y., Wan Y., He H. Deep Reinforcement Learning Based Task Offloading
for UAV-Assisted Edge Computing // IEEE International Conference on Unmanned Systems
(ICUS), 2023: IEEE. – P. 1104-1111.
29. Ding Y., Yang Z., Pham Q.V., Hu Y., Zhang Z., Shikh-Bahaei M. Distributed machine learning for
UAV swarms: computing, sensing, and semantics // IEEE Internet Things J. – 2023. – DOI: 10. 1109/
JIOT. 2023. 33413 07.
30. Sutton R.S., Barto A.G. Reinforcement Learning: An Introduction. A Bradford Book. – 2018. – 526 p.
