NATURAL LANGUAGE CONTROL OF CONSTRUCTION ROBOTIC SYSTEMS
Abstract
The study aims to investigate the potential of natural language control systems for construction robots. It is the lack of reliable natural language processing systems that serves as a limiting factor that prevents intelligent robotics from fully realizing its potential. The work provides an overview of modern robotic construction systems that are used to facilitate and improve construction and engineering processes and tasks. What unites all these systems is the lack of natural language control. In this paper, we present principles, algorithms, and methods that allow an intelligent agent to penetrate the essence of the context of a situation unfolding in the field of construction and engineering tasks. The approach is based on a multi-agent neurocognitive architecture, which serves as a kind of tool for modeling the process of automatic interpretation of phrases taken from a limited subset of natural language. In order for an intelligent agent to correctly interpret an incoming message, it must accurately determine the conditions, actions, properties, and relationships that take place in the "intelligent agent - environment" system. Only then does the agent gain the ability to interpret the context of the current dialogue and generate statements necessary for designing cooperative behavior aimed at jointly overcoming technical obstacles. One of the most common problems requiring a solution in the rapidly developing field of robotics is the development of a dialogue control system capable of coordinating joint human-machine behavior and interpreting goals and mission conditions set out in natural language. A control system based on natural language is an integral part of an intelligent system, the foundation of which is a self-organizing multi-agent neurocognitive architecture. Its main goal is to establish seamless communication between human-machine teams so that they can jointly set, describe and successfully complete complex construction tasks. The fundamental element of the approach is multi-agency, which allows the robot's decision-making system to be flexible, adaptive and continuously expand the range of its knowledge, generating questions necessary for further work.
References
1. Liu Y. et al. Robotics in the construction sector: Trends, advances, and challenges, Journal of Intelligent & Robotic Systems, 2024, Vol. 110, No. 2, pp. 72.
2. Rathore M.M. et al. The role of AI, machine learning, and big data in digital twinning: A systematic literature review, challenges, and opportunities, IEEE Access, 2021, Vol. 9, pp. 32030-32052.
3. Aghimien D.O. et al. Mapping out research focus for robotics and automation research in construction-related studies: A bibliometric approach, Journal of Engineering, Design and Technology, 2020,
Vol. 18, No. 5, pp. 1063-1079.
4. Li G. et al. Skin-inspired quadruple tactile sensors integrated on a robot hand enable object recognition, Science Robotics, 2020, Vol. 5, No. 49, pp. eabc8134.
5. Suzumori K., Faudzi A.A. Trends in hydraulic actuators and components in legged and tough robots: a review, Advanced Robotics, 2018, Vol. 32, No. 9, pp. 458-476.
6. Zhang J., Wang M. A survey on robots controlled by motor imagery brain-computer interfaces, Cogni-tive Robotics, 2021, Vol. 1, pp. 12-24.
7. Akinlolu M. et al. A bibliometric review of the status and emerging research trends in construction safety management technologies, International Journal of Construction Management, 2022, Vol. 22, No. 14, pp. 2699-2711.
8. Akinosho T.D. et al. Deep learning in the construction industry: A review of present status and future innovations, Journal of Building Engineering, 2020, Vol. 32, pp. 101827.
9. Gharbia M. et al. Robotic technologies for on-site building construction: A systematic review, Journal of Building Engineering. 2020. Vol. 32. [[. 101584.
10. Maskuriy R. et al. Industry 4.0 for the construction industry: Review of management perspective, Econ-omies, 2019, Vol. 7, No. 3, pp. 68.
11. Forcael E. et al. Construction 4.0: A literature review, Sustainability, 2020, Vol. 12, No. 22, pp. 9755.
12. Biswal P., Mohanty P.K. Development of quadruped walking robots: A review, Ain Shams Engineering Journal, 2021, Vol. 12, No. 2, pp. 2017-2031.
13. Yogesh G. Construction Robotics: Revolutionizing the Building Industry, Journal of Architectural Engi-neering Technology, 2024, Vol. 13, No. 418.
14. Available at: https://cdn.thomasnet.com/ccp/00142951/263811.pdf (accessed 10 September 2024).
15. Available at: https://iconbuild.com/robotics (accessed 19 October 2024).
16. Available at: https://apis-cor.com/ (accessed 10 February 2025).
17. Available at: https://www.fronius.com/en/welding-technology/product-information/welding-automation/robotic-welding (accessed 15 December 2024).
18. Available at: https://www.brokk.kz/ (accessed 23 November 2024).
19. Stenman M. Automatic speech recognition. An evaluation of Google Speechб 2015.
20. "Cloud Speech-to-Text," Google. Available at: https://cloud.google.com/speech-to-text/ (accessed 19 January 2022).
21. Reis A., Paulino D., Paredes H., Barroso I., Monteiro M.J., Rodrigues V. Using intelligent personal assistants to assist the elderlies an evaluation of Amazon Alexa, Google Assistant, Microsoft Cortana, and Apple Siri, 2-nd International Conference on Technology and Innovation in Sports, Health and Wellbeing (TISHW), 2018, pp. 1-5.
22. Brill T., Munoz L., Richard J. Siri, Alexa, and other digital assistants: A study of customer satisfaction with artificial intelligence applications, Journal of Marketing Management, 2019, Vol. 35, No. 15-16, pp. 1401-1436.
23. Tulshan A.S., Dhage S.N. Survey on Virtual Assistant: Google Assistant, Siri, Cortana, Alexa, Commu-nications in Computer and Information Science, 2019, 968, pp. 190-201.
24. Gavrilovich N.V., Seytvelieva S.N. Analiz kommercheskikh sistem raspoznavaniya rechi s otkrytym API [Analysis of commercial speech recognition systems with an open API], Tavricheskiy nauchnyy oboz-revatel' [Tavrichesky Scientific Observer], 2016, No. 6 (11). Available at: https://cyberleninka.ru/article/n/analiz-kommercheskih-sistem-raspoznavaniya-rechi-s-otkrytym-api (accessed 23 January 2022).
25. Microsoft Corporation. Exploring New Speech Recognition and Synthesis APIs in Windows Vista. Microsoft. Available at: https://learn.microsoft.com/en-us/archive/msdn-magazine/2006/january/exploring-speech-recognition-and-synthesis-apis-in-windows-vista (accessed 24 April 2022).
26. Rodemann T. Towards Speech Acquisition in Natural Interaction on ASIMO, Journal of the Robotics Society of Japan, 2010, 28.1, pp. 18-22.
27. Heracleous P., Yoneyama A. A comprehensive study on bilingual and multilingual speech emotion recognition using a two-pass classification scheme, PloS one, 2019, Vol. 14, No. 8, pp. 220-386.
28. Bogdanov A., Dudorov E., Kutlubaev I., Permyakov A., Pronin A. Control System of a Manipulator of the Anthropomorphic Robot FEDOR, 12th International Conference on Developments in e-Systems En-gineering. IEEE, INSPEC Accession Number 9557273, 2019, pp. 449-453.
29. Stolcke A., Droppo. J. Comparing Human and Machine Errors in Conversational Speech Transcription, Interspeech, 2017, pp. 137-141.
30. Saon G., Kurata G., Sercu T., Audhkhasi K.,Thomas S., Dimitriadis D., Cui X., Ramabhadran B., Picheny M., Lim L.-L., Roomi B., Hall P. English Conversational Telephone Speech Recognition by Humans and Machines, INTERSPEECH, 2017
31. Glenn M.L. et al. Transcription Methods for Consistency, Volume and Efficiency, LREC, 2010.
32. Marti A., Cobos M., Lopez J. Automatic Speech Recognition in Cocktail-Party Situations: A specific Training for Separated Speech, The Journal of the Acoustical Society of America, 2012, pp. 1529-1535.
33. Golumbic E.M. Z. et al. Mechanisms underlying selective neuronal tracking of attended speech at a “cocktail party”, Neuron, 2013, Vol. 77, No. 5, pp. 980-991.
34. Nagoev Z., Gurtueva I., Malyshev D., Sundukov Z. Multi-agent Algorithm Imitating Formation of Pho-nemic Awareness, In: Samsonovich A. (eds) Biologically Inspired Cognitive Architectures BICA 2019. Advances in Intelligent Systems and Computing, 2020, pp. 364-369.
35. Nagoev Z., Gurtueva I., Anchekov M. Generalized Structure of Active Speech Perception Based on Mul-tiagent Intelligence, Studies in Computational Intelligence, 2022, pp. 319-326
36. Nagoev Z., Nagoeva O., Pshenokova I., Bzhikhatlov K., Gurtueva I., Kankulov S. Multi-agent neural-like models for the integration of multimodal medical examination data, BECB, 2022.
37. Nagoev Z.V., Nagoeva O.V. Obosnovanie simvolov i mul'tiagentnye neyrokognitivnye modeli semantiki estestvennogo yazyka Obosnovanie simvolov i mul'tiagentnye neyrokognitivnye modeli semantiki estestvennogo yazyka [Justification of Symbols and Multi-Agent Neurocognitive Models of Natural Language Semantics. Justification of Symbols and Multi-Agent Neurocognitive Models of Natural Lan-guage Semantics]. Nal'chik: Izd-vo KBNTS RAN, 2022, 150 p.
38. Makoeva D., Nagoeva O., Gurtueva I. Formal Representation of natural language elements in multi-agent system based of self-organization of distributed neurocognitive architectures, Procedia Computer Science, 2022, Vol. 213, pp. 631-635.
39. Anchekov M.I., Bzhikhatlov K.Ch., Kankulov S.A., Nagoev Z.V., Nagoeva O.V. Mul'tiagentnyy algoritm obosnovaniya simvolov konventsional'nogo yazyka na osnove situativno obuslovlennogo razvitiya ney-rokognitivnoy arkhitektury [Multi-agent algorithm for substantiating conventional language symbols based on situationally determined development of neurocognitive architecture], Izvestiya Kabardino-Balkarskogo nauchnogo tsentra RAN [Bulletin of the Kabardino-Balkarian Scientific Center of the Rus-sian Academy of Sciences], 2022, No. 6 (110), pp. 48-60.
40. Anchekov M.I., Bzhikhatlov K.CH., Nagoev Z.V., Pshenokova I.A. Ontoepisotsiofilogeneticheskoe razvi-tie sistem obshchego iskusstvennogo intellekta na osnove mul'tiagentnykh neyrokognitivnykh arkhitektur [Ontoepisociophylogenetic development of general artificial intelligence systems based on multi-agent neurocognitive architectures], Izvestiya Kabardino-Balkarskogo nauchnogo tsentra RAN [Bulletin of the Kabardino-Balkarian Scientific Center of the Russian Academy of Sciences], 2022, No. 6 (110), pp. 61-75.
41. Nagoev Z.V. Intellektika, ili Myshlenie v zhivykh i iskusstvennykh sistemakh [Intelligence, or Thinking in living and artificial systems]. Nal'chik: Izd-vo KBNTS RAN, 2013, pp. 16.
42. Pshenokova I.A., Bzhikhatlov K.Ch., Ksalov A.M., Zammoev A.U. Intellektual'naya sistema prinyatiya resheniy dlya aktivnoy zashchity rasteniy [Intelligent Decision-Making System for Active Plant Protec-tion], Informatsionnoe obshchestvo [Information Society], 2023, No. 3, pp. 38-46.
