COMPREHENSIVE APPROACH TO INTRUDER RECOGNITION BASED ON VIDEO IMAGERY
Abstract
Intruder recognition in uncontrolled environments is a critical function of biometric-based security control systems (SCS), ensuring protection at facilities with large crowds. The accuracy of such systems can be improved by combining multiple biometric traits extracted from video imagery. However, processing video data involves several challenges, including viewpoint variation, occlusion, and selecting an appropriate feature fusion level. To resolve these challenges, a complete approach to intruder recognition is proposed. The approach is based on video preprocessing, the combination of two convolutional neural networks (CNNs), Dempster–Shafer theory, and the random forest method. These techniques fuse behavioral biometric features at the score level to classify video sequences. Gait and gestural behavior are selected as biometric modalities, as they can be captured without direct subject interaction. For the classification task, three classes of video data are defined: class 1 – person is not intruder, class 2 – a potential intruder, class 3 – intruder. The study also presents a general architecture for the proposed approach, along with a detailed description of its processing stages. The effectiveness of the approach is measured through experiments performed on the KTH dataset, which comprises six types of simple human actions performed by different subjects under different background conditions. Experimental results show that the proposed approach improves intruder recognition accuracy in uncontrolled environments, achieving an 87 % classification rate.
##article.references##
1. Jain Anil, Nandakumar Karthik, Ross Arun 50 years of biometric research: accomplishments, challeng-es, and opportunities, Pattern Recognition Letters, 2016, Vol. 79, pp. 80-105. DOI: 10.1016/j.patrec.2015.12.013.
2. Kolodenkova A.E. Ontologiya identifikatsii narushitelya po dvizheniyam tela i litsu v video-nablyudeniyakh [Ontology of human identification by face and body motions in video surveillance sys-tems], Ontologiya proektirovaniya [Ontology of Designing], 2023, No. 1, pp. 55-74. DOI: https://doi.org/10.18287/2223-9537-2023-13-1-55-74.
3. Pahuja S., Goel N. Multimodal biometric authentication: A review, AI Communications, 2024,
Vol. 37, No. 4, pp. 525-547. DOI: 10.3233/AIC-220247.
4. Xia F., Wei X., Q. Zhao, Jin B., Yu Z., Zha Z. Exploration on the application of multimodal biometric feature recognition in system access control, 2024 IEEE 13th International Conference on Communica-tion Systems and Network Technologies (CSNT), 2024, pp. 1-6. DOI: 10.1109/CSNT60213.2024.10545831.
5. Orlov A.A., Mironov M.I., Abramova E.S. Obzor i analiz podkhodov i prakticheskikh oblastey prime-neniya raspoznavaniya emotsiy cheloveka [Review and analysis of approaches and practical applications of human emotion recognition], Vestnik Yuzhno-Ural'skogo gosudarstvennogo universiteta. Seriya: Komp'yuternye tekhnologii, upravlenie, radioelektronika [Bulletin of the South Ural State University. Series: Computer Technologies, Automatic Control, Radio Electronics], 2023, Vol. 23, No. 4, pp. 5-15. DOI: 10.14529/ctcr230401.
6. Rangel G., Cuevas-Tello J.C., Nunez-Varela J., Puente C., Silva-Trujillo A.G. A survey on convolution-al neural networks and their performance limitations in image recognition tasks, Journal of sensors, 2024, No. 1, pp. 1-29. DOI: 10.1155/2024/2797320.
7. Safonov A.A., Bulgakov V.G., Varchenko I.A. Kriminalisticheskoe issledovanie dinamicheskikh prizna-kov cheloveka: istoriya i sovremennoe sostoyanie [Criminalistic investigation of dynamic human charac-teristics: history and current state], Obshchestvo i pravo [Society and Law], 2010, No. 3, pp. 250-257.
8. Kobets P.N. O kompleksnom izuchenii lichnosti prestupnika v otechestvennoy kriminologii. Problemy razvitiya lichnosti [On the complex study of the personality of a criminal in domestic criminology. Prob-lems of personality development], Mater. mezhdunarod. nauch.-prakt. konf. Praga, 2013 [Mater. inter-national scientific-practical. conf. Prague, 2013], pp. 93-95.
9. Sprager S., Juric M.B. Inertial sensor-based gait recognition: A Review, Sensors, 2015, Vol. 15,
pp. 22089-22127. DOI: 10.3390/s150922089.
10. Hou L., Wan W., Han K., Muhammad R., Yang M. Human detection and tracking over camera net-works: A review, 2016 International Conference on Audio, Language and Image Processing (ICALIP), 2016, pp. 574-580. DOI: 10.1109/ICALIP.2016.7846643.
11. Li W., Hou S., Zhang C., Cao C., Liu X., Huang Y., Zhao Y. An in-depth exploration of person re-identification and gait recognition in cloth-changing conditions, Proceedings of the IEEE/CVF Confer-ence on Computer Vision and Pattern Recognition, 2023, pp. 13824-13833. DOI: 10.1109/CVPR52729.2023.01328.
12. Fedyunina A.P. Vyyavlenie kharakterologicheskikh priznakov i sostavlenie psikhologicheskogo portreta vozmozhnogo narushitelya i loyal'nogo sotrudnika v sfere informatsionnoy bezopasnosti [Identification of characterological signs and drawing up a psychological portrait of a possible violator and a loyal em-ployee in the field of information security], Vestnik AGTU [Bulletin of AGTU], 2007, No. 4, pp. 231-236.
13. Al-Azawi R.J. Enhancing recognition accuracy and efficiency through intelligent frame selection in un-controlled conditions, J Opt., 2024, pp. 1-7. DOI: https://doi.org/10.1007/s12596-024-02191-4.
14. Zhao L. A Hybrid deep learning-based intelligent system for sports action recognition via visual knowledge discovery, IEEE Acces., 2023, Vol. 11, pp. 46541-46549. DOI: 10.1109/ACCESS.2023.3275012.
15. Zhou J., Jiangbo Ai. Zheng Wang, Chen S., Wei Q. Discovering attractive segments in the user generated video streams, Lecture Notes in Computer Science, 2019, Vol. 11642, pp. 236-250. DOI: https://doi.org/10.1007/978-3-030-26075-0_18.
16. Mingju Chen, Han Xiaofeng, Zhang Hua, Lin Guojun, Kamruzzaman M.M. Quality-guided key frames selection from video stream based on object detection, Journal of Visual Communication and Image Representation, 2019, Vol. 65, pp. 1-7. DOI: https://doi.org/10.1016/j.jvcir.2019.102678.
17. Gharabagh Abdorreza, Hajihashemi Vahid, Ferreira Marta, Machado José, Tavares Joao Best Frame Selection to Enhance Training Step Efficiency in Video-Based Human Action Recognition, Applied Sci-ences, 2022, Vol. 12, pp. 1-12. DOI: 10.3390/app12041830.
18. Trofimenko Ya.M., Vinogradova L.N., Ershov E.V. Algoritmy predobrabotki izobrazheniy v sisteme identifikatsii lits v videopotoke [Algorithms for image preprocessing in the face identification system in the video stream], Vestnik Cherepovetskogo gosudarstvennogo universiteta [Bulletin of the Cherepovets State University], 2019, No. 4, pp. 21-29. DOI: 10.23859/1994-0637-2019-4-91-2.
19. Kumari P., Reddy S.R.N., Yadav R. Application of Dempster-Shafer Theory in Sensor Data Fusion, Proceedings of the International Conference on Machine Learning, Deep Learning and Computational Intelligence for Wireless Communication, 2024, pp. 91-102. DOI: https://doi.org/10.1007/978-3-031-47942-7_9.
20. Recognition of human actions. Available at: https://www.csc.kth.se/cvap/actions/ (accessed 24 February 2026).
21. Computer-aided identification of degenerative neuromuscular diseases based on gait dynamics and en-semble decision tree classifiers. Access mode: https://journals.plos.org/plosone/article?id=10.1371/ jour-nal.pone.0252380 (accessed 24 February 2026).
22. Zhang Z. A gait recognition method for a moving target image in sports based on a decision tree, Inter-national Journal of Biometrics, 2021, Vol. 13, No. 2, pp. 165-179. DOI: 10.1504/IJBM.2021.114642.
