METHODS AND MEANS OF TRACKING THE MOVEMENT AND INTERACTION OF EMPLOYEES AND CUSTOMERS BY VIDEO IMAGE
Abstract
Due to the rapid development of the sphere of trade, the means of automatic control of the
work of employees providing services to customers are gaining particular popularity. At the moment,
there are many modern approaches, methods and algorithms for automatically tracking
buyers and sellers in the store. Modern companies are trying to solve this problem in different
ways: counting visitors, monitoring devices, various neural network solutions, and so on. After
reviewing the solutions with the necessary functionality, the main disadvantages were identified,
such as, for example, high cost, inconvenience in use, and so on. As a result, the authors set a
goal: to improve the quality of tracking the movement of employees / customers through the development
of automated means and methods of movement control, inter-chamber tracking and identification
of the individual. The article describes a method for automatic recognition and tracking of
employees of stores and firms. The method is based on a cascade of neural networks and algorithms
that allow recognizing customers and employees in uniform, as well as evaluating the
quality of employees' work and customer satisfaction by voice. As the results of the research, this
article presents models and methods for classifying customers and sellers by uniform, methods for
determining the level of interaction between sellers and customers based on algorithms for determining
the satisfaction of visitors and customers by voice and face, and algorithms for determining
the quality of employees' work. The developed methods can improve the efficiency of employees, as
well as increase the quality of services provided. Based on the results of the work, testing was
carried out and a conclusion was made about the satisfactory performance of the presented methods
and algorithms.
References
sistema kontrolya aktivnosti pokupateley v magazine s modulyami otsenki raboty ego sotrudnikov
[Automatic system for monitoring customer activity in a store with modules for evaluating the work
of its employees], Prikaspiyskiy zhurnal: upravlenie i vysokie tekhnologii [Caspian Journal: Management
and High technologies], 2020, No. 2 (50), pp. 22-32.
2. Ul'yanova O.A. Psikhologicheskie osobennosti prodavtsov-konsul'tantov setevogo marketinga
[Psychological features of sales consultants of network marketing], Vestnik Samarskoy
gumanitarnoy akademii. Seriya: Psikhologiya [Bulletin of the Samara Humanitarian Academy.
Series: Psychology], 2013, No. 1, pp. 27-41.
3. Ghosh G., Swarnalatha K.S. A Detail Analysis and Implementation of Haar Cascade Classifier,
Advances in Intelligent Systems and Computing, 2021, pp. 341-59.
4. Bharathi P., Spurthy B.L.D., Bhavani Priya A., Yashvardhan Krishna Murthy B., Charan Sai
Varaha Teja Ch.. Real-Time Object Detection for Blind People, International Journal of Advanced
Research in Science, Communication and Technology, 2023, pp. 262-266.
5. Bobkov A.S., Rozaliev V.L. Fazifikatsiya dannykh, opisyvayushchikh dvizhenie cheloveka
[Fuzzification of data describing human movement], Otkrytye semanticheskie tekhnologii
proektirovaniya intellektual'nykh sistem (OSTIS-2011): Mater. mezhdunar. nauch.-tekhn. konf.
[Open semantic technologies for designing intelligent systems (OSTIS-2011): Proceedings of
the International Scientific and Technical Conference]. Minsk, 2011, pp. 483-486.
6. Rozaliev V.L., Orlova Yu.A. Opredelenie dvizheniy i poz dlya identifikatsii emotsional'nykh
reaktsiy cheloveka [Determination of movements and poses to identify emotional reactions of
a person], 11-aya Mezhdunarodnaya konferentsiya po raspoznovaniyu ob-razov i analizu
izobrazheniy: novye informatsionnye tekhnologii (PRIA-11-2013) [11th International Conference
on Pattern Recognition and Image Analysis: New Information Technologies (PRIA-11-
2013)]. Samara, 2013, No. 2, pp. 713-716.
7. Samyak S., Gupta A., Raj T., Karnam A., Mamatha HR. Speech Emotion Analyzer, Innovative
Data Communication Technologies and Application, 2022, pp. 113-124.
8. Sharmila Sk., Nagasai GK., Sowmya M., Prasanna AS., Sri SN., Meghana N. Automatic Attendance
System based on FaceRecognition using Machine Learning, 2023 7th International
Conference on Computing Methodologies and Communication (ICCMC), 2023.
9. Xia J., Zhang H., Wen S., Yang S., Xu M. An Efficient Multitask Neural Network for Face
Alignment, Head Pose Estimation and Face Tracking, Expert Syst., 2021, Vol. 205, pp. 117368.
10. Khorunzhiy M.D. Metod kolichestvennoy otsenki tsvetov razlichiy pri vospriyatii tsifrovykh
izobrazheniy [A method for quantifying the colors of differences in the perception of digital
images], Vestnik NGU. Seriya: Informatsionnye tekhnologii [Bulletin of the NSU. Series: Information
Technology], 2008, No. 1, pp. 9.
11. Iqbal U., Gall J. Multi-person Pose Estimation with Local Joint-to-Person Associations, European
Conference on Computer Vision (ECCV) Workshops, Crowd Understanding. Amsterdam, 2016.
12. Samoylov A.N., Sergeev N.E., Dayebal D.B., Kucherova M.S. Model' intellektual'noy
informatsionnoy sistemy dlya raspoznavaniya pol'zovateley sotsial'noy seti s ispol'zovaniem
bioinspirirovannykh metodov [A model of an intelligent information system for recognizing
social network users using bioinspired methods], Inzhenernyy vestnik Dona [Engineering Bulletin
of the Don.], 2018, No. 4 (51), pp. 137.
13. Ul'ev A.D. Sistema dlya opredeleniya udovletvorennosti klienta magazina okazannymi
uslugami [A system for determining the satisfaction of the store's customer with the services
rendered], NovaUm.Ru, 2022, No. 37, pp. 4-8.
14. Nasr M., Osama R., Ayman H., Mosaad N., Ebrahim А. Realtime Multi-Person 2D Pose Estimation,
Int. J. Advanced Networking and Applications, 2020, No. 11, pp. 4501-4508.
15. Osipova Yu.A., Lavrov D.N. Primenenie klasternogo analiza metodom k-srednikh dlya
klassifikatsii tekstov nauchnoy napravlennosti [Application of cluster analysis by the k-means
method for classification of scientific texts], MSiM [MSiM], 2017, No. 3 (43), pp. 14.
16. Andronova N.E., Grebenyuk P.E., Chmutin A.M. Algoritm i programmnaya realizatsiya
upravleniya ottenochnym kontrastom tsifrovykh izobrazheniy [Algorithm and software implementation
of the control of the hue contrast of digital images], Inzhenernyy vestnik Dona
[Engineering Bulletin of the Don], 2016, No. 4 (43).
17. Ul'ev A.D., Rozaliev V.L., Orlova Yu.A., Alekseev A.V. Avtomaticheskiy kontrol' urovnya
okazaniya uslug prodavtsom-konsul'tantom [Automatic control of the level of services rendered
by a sales consultant], Iskusstvennyy intellekt v reshenii aktual'nykh sotsial'nykh i
ekonomicheskikh problem XXI veka: Sb. statey po materialam Chetvertoy vserossiyskoy
nauchno-prakticheskoy konferentsii, provodimoy v ramkakh Permskogo estestvennonauchnogo
foruma «Matematika i global'nye vyzovy XXI veka», Perm', 21–23 maya 2019 goda [Artificial
intelligence in solving urgent social and economic problems of the XXI century: A collection
of articles based on the materials of the Fourth All-Russian Scientific and Practical Conference
held within the framework of the Perm Natural Science Forum "Mathematics and Global Challenges
of the XXI Century", Perm, May 21-23, 2019]. Vol. Part I. Perm', 2019, pp. 145-149.
18. Cao Z., Simon T., Wei S., Sheikh Y. Realtime Multi-person 2D Pose Estimation Using Part
Affinity Fields, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
2016, pp. 1302-1310.
19. Park C., Lee H.S., Kim W.J., Bae H.B., Lee J., Lee S. An Efficient Approach Using Knowledge
Distillation Methods to Stabilize Performance in a Lightweight Top-Down Posture Estimation
Network, Sensors, 2021, 21 (22), pp. 7640. Available at: https://doi.org/10.3390/s21227640.
20. Aonty S.S., Deb K., Sarma M. S., Dhar P.K., Shimamura T. Multi-Person Pose Estimation
Using Group-Based Convolutional Neural Network Model, in IEEE Access, 2023, Vol. 11,
pp. 42343-42360. DOI: 10.1109/ACCESS.2023.3271593.
