ALGORITHM FOR SIGN LANGUAGE RECOGNITION USING CONVOLUTIONAL NEURAL NETWORK

Abstract

This paper addresses the problem of automatic recognition of Russian Sign Language (RSL) using computer vision and deep learning methods. The relevance of the study is driven by a steady increase in the number of people with hearing impairments: according to the World Health Organization, there are currently about 70 million deaf and hard-of-hearing individuals worldwide, and this number is projected to reach 630 million by 2035. The development of effective gesture recognition algorithms is an important direction for creating contactless human–machine interaction systems aimed at improving accessibility of information technologies and enhancing the quality of life for people with hearing disabilities. The aim of the study is to develop and experimentally validate an algorithm for real-time recognition of Russian Sign Language alphabet gestures in a video stream using a convolutional neural network. A specialized dataset was created, consisting of 430 images of hand gestures corresponding to the letters of the RSL alphabet, captured from different angles and under varying lighting conditions. The model was implemented using TensorFlow and Keras libraries, while integration with the video stream was performed using OpenCV and a marker-based hand tracking system. As a result of training and testing, the proposed model achieved a recognition accuracy of 99% on the test dataset. A comparative analysis with classical machine learning methods demonstrated the superiority of the convolutional neural network in terms of classification accuracy and robustness to external noise. The obtained results confirm the effectiveness of the proposed approach and its applicability for real-time systems intended for communication, educational, and rehabilitation applications, as well as for the development of advanced human–machine interaction interfaces.