METHOD FOR RECOGNIZING TEXT DATA IN IMAGES
Keywords:
OCR, video stream processing, text information extraction, classification, image recognition, labeling
Abstract
The purpose of the study is to study the problems arising in the process of digital image
processing in systems for obtaining textual characteristics of product objects. Such as the selection
of objects that fall into the frames of the video stream and the recognition of text markings,
without the use of specialized hardware. In particular, the problems that arise when working
with images containing different levels of noise and distortion. The objectives of the study i nclude
a comparative and analytical examination of diverse methods and algorithms utilized in
the realm of digital image processing. The primary objectives include identifying, segmenting,
and classifying text-containing portions within the video stream. The study aims to construct a
mathematical model for text extraction from video frames that is adaptable to a wide array of
objects. Evaluate recognition accuracy under varying levels of noise and perform a comparative
analysis against alternative solutions based on the acquired data. The presented system
analyzes the frames of the video stream and classifies the characteristics of the products in the
frame. The solution demonstrates the behaviour and capabilities of digital image processing
methods in various conditions in relation to the tasks of text classification and object search in
a video stream. During the development of this system, a comparison of various options for recognizing
symbolic information was carried out.
References
1. Kopylov I.V., Kazakov A.V., Malygin L.L. Identifikatsiya markirovok rulonov metalloprokata
[Identification of markings of rolled metal rolls], Vestnik cherepovetskogo gosudarstvennogo
universiteta [Bulletin of Cherepovets State University], 2016, No. 5 (74), pp. 12-15.
2. Hsu C., Yang Y., Chang A., Morsalin S., Shen G., Shiu L. Automatic Recognition of Container
Serial Code, 2023 International Conference on Consumer Electronics, 2023, pp. 257-258.
3. Feng W., He W., Yin F., Zhang X., Liu C. TextDragon: An End-to-End Framework for Arbitrary
Shaped Text Spotting, International Conference on Computer Vision (ICCV), 2019,
pp. 9075-9084.
4. Yazdi M., Bouwmans T. New Trends on Moving Object Detection in Video Images Captured
by a moving Camera: A Survey, Computer Science Review, 2018, No. 28, pp. 157-177.
5. Provotorov A.V., Orlov A.A. Metodika poetapnogo obnaruzheniya markirovki slyabov [Methodology
for step-by-step detection of slab markings], Sovremennye problemy nauki i
obrazovaniya [Modern problems of science and education], 2012, No. 6, pp. 98-98.
6. Pyzh S.V., Ganicheva O.G. Mobil'naya avtomatizirovannaya sistema inventarizatsii
metallurgicheskoy produktsii i obespecheniya bezopasnosti v skladskikh pomeshcheniyakh
[Mobile automated system for inventory of metallurgical products and security in warehouses],
Nauchno-tekhnicheskiy progress v chernoy metallurgii: Sb. statey [Scientific and technical
progress in ferrous metallurgy: Collection of articles]. Cherepovets, 2013, pp. 266-271.
7. Xiang Z., You Z., Qian M., Zhang J. Metal stamping character recognition algorithm based on
multi-directional illumination image fusion enhancement technology, EURASIP Journal on
Image and Video Processing, 2018, Vol. 18, pp. 1-11. DOI: 10.1186/s13640-018-0321-7.
8. Zakharov A.A., Zhiznyakov A.L., Titov V.S. Metod nakhozhdeniya sootvetstviy na
izobrazheniyakh s ispol'zovaniem struktur [Method for finding correspondences in images using
structures], Komp'yuternaya optika [Computer Optics], 2019, Vol. 43, No. 5, pp. 810-817.
– DOI: 10.18287/2412-6179-2019-43-5-810-817.
9. Goremykin I.V., Butenko L.N. Deskriptory tsvetovykh regionov v zadache obespecheniya
tsvetovoy soglasovannosti izobrazheniya [Descriptors of color regions in the problem of ensuring
color consistency of an image], Perspektivy razvitiya informatsionnykh tekhnologiy [Prospects
for the development of information technologies], 2015, No. 27, pp. 6-17.
10. Silakov N.V., Tassov K.L. Obzor algoritmov obnaruzheniya tekstovykh oblastey na kadrakh
videopotoka [Review of algorithms for detecting text areas on video stream frames], Vestnik
RGGU. Seriya: Informatika. Informatsionnaya bezopasnost'. Matematika [Bulletin of the Russian
State University for the Humanities. Series: Computer Science. Information Security.
Mathematics], 2020, No. 2, pp. 27-45. DOI: 10.28995/2686-679X-2020-2-27-45.
11. Mahajan S., Rani R., Trehan K. DELIGHT-Net: DEep and LIGHTweight network to segment
Indian text at word level from wild scenic images, International Journal of Multimedia Information
Retrieval, 2023, No. 12 (2), pp. 29-37.
12. Gonsales R., Vuds R. TSifrovaya obrabotka izobrazheniy [Digital image processing]. Moscow:
Tekhnosfera, 2012, 1104 p.
13. Shapiro L., Stokman Dzh. Komp'yuternoe zrenie [Computer vision]. Moscow: BINOM, 2013,
752 p.
14. Khomonenko A.D., Yakovlev E.L. Obosnovanie arkhitektury svertochnoy neyronnoy seti dlya
avtonomnogo raspoznavaniya ob"ektov na izobrazheniyakh bortovoy vychislitel'noy sistemoy
[Justification of the architecture of a convolutional neural network for autonomous recognition
of objects in images by an on-board computer system], Naukoemkie tekhnologii v
kosmicheskikh issledovaniyakh Zemli [Science-intensive technologies in space research of the
Earth], 2018, Vol. 10, No. 6, pp. 86-93.
15. Busta M., Neumann L., Matas J. Deep TextSpotter: An End-to-End Trainable Scene Text Localization
and Recognition Framework, 2017 IEEE International Conference on Computer Vision
(ICCV), 2017, pp. 2223-2231.
16. Liu Y., Wang Y., Shi H. A Convolutional Recurrent Neural-Network-Based Machine Learning
for Scene Text Recognition Application. Symmetry, Symmetry, 2023, No. 15 (4), 849.
17. Liu X., Liang D., Yan S., Chen D., Qiao Y., Yan J. FOTS: Fast Oriented Text Spotting with a
Unified Network, Computer Vision and Pattern Recognition, 2018, pp. 5676-5685.
18. Chaitra Y., Dinesh R., Gopalakrishna M., Prakash B.A. Deep-CNNTL: Text Localization from
Natural Scene Images Using Deep Convolution Neural Network with Transfer Learning, Arabian
Journal for Science and Engineering, 2021, No. 47 (9), pp. 9629-9640.
19. He T., Tian Z., Huang W., Shen C., Qiao Y., Sun C. An End-to-End TextSpotter with Explicit
Alignment and Attention, IEEE Conference on Computer Vision and Pattern Recognition,
2018, pp. 5020-5029.
20. Bulatov K.B. A method to reduce errors of string recognition based on combination of several
recognition results with per-character alternatives, Vestnik Yuzhno-Ural'skogo gosudarstvennogo
universiteta. Seriya: Matematicheskoe modelirovanie i programmirovanie [Bulletin
of the South Ural State University. Series: Mathematical modeling and programming], 2019,
Vol. 12, No. 3, pp. 74-88. DOI: 10.14529/mmp190307.
21. Panishchev V.S., Trufanov M.I., Dobroserdov O.G., Khomyakov O.O. Raspoznavanie simvol'noy
informatsii dlya avtomatizatsii proizvodstvennykh protsessov [Recognition of symbolic information
for automation of production processes], Izvestiya Yugo-Zapadnogo gosudarstvennogo universiteta
[News of the South-Western State University], 2021, No. 1, pp. 122-137.
[Identification of markings of rolled metal rolls], Vestnik cherepovetskogo gosudarstvennogo
universiteta [Bulletin of Cherepovets State University], 2016, No. 5 (74), pp. 12-15.
2. Hsu C., Yang Y., Chang A., Morsalin S., Shen G., Shiu L. Automatic Recognition of Container
Serial Code, 2023 International Conference on Consumer Electronics, 2023, pp. 257-258.
3. Feng W., He W., Yin F., Zhang X., Liu C. TextDragon: An End-to-End Framework for Arbitrary
Shaped Text Spotting, International Conference on Computer Vision (ICCV), 2019,
pp. 9075-9084.
4. Yazdi M., Bouwmans T. New Trends on Moving Object Detection in Video Images Captured
by a moving Camera: A Survey, Computer Science Review, 2018, No. 28, pp. 157-177.
5. Provotorov A.V., Orlov A.A. Metodika poetapnogo obnaruzheniya markirovki slyabov [Methodology
for step-by-step detection of slab markings], Sovremennye problemy nauki i
obrazovaniya [Modern problems of science and education], 2012, No. 6, pp. 98-98.
6. Pyzh S.V., Ganicheva O.G. Mobil'naya avtomatizirovannaya sistema inventarizatsii
metallurgicheskoy produktsii i obespecheniya bezopasnosti v skladskikh pomeshcheniyakh
[Mobile automated system for inventory of metallurgical products and security in warehouses],
Nauchno-tekhnicheskiy progress v chernoy metallurgii: Sb. statey [Scientific and technical
progress in ferrous metallurgy: Collection of articles]. Cherepovets, 2013, pp. 266-271.
7. Xiang Z., You Z., Qian M., Zhang J. Metal stamping character recognition algorithm based on
multi-directional illumination image fusion enhancement technology, EURASIP Journal on
Image and Video Processing, 2018, Vol. 18, pp. 1-11. DOI: 10.1186/s13640-018-0321-7.
8. Zakharov A.A., Zhiznyakov A.L., Titov V.S. Metod nakhozhdeniya sootvetstviy na
izobrazheniyakh s ispol'zovaniem struktur [Method for finding correspondences in images using
structures], Komp'yuternaya optika [Computer Optics], 2019, Vol. 43, No. 5, pp. 810-817.
– DOI: 10.18287/2412-6179-2019-43-5-810-817.
9. Goremykin I.V., Butenko L.N. Deskriptory tsvetovykh regionov v zadache obespecheniya
tsvetovoy soglasovannosti izobrazheniya [Descriptors of color regions in the problem of ensuring
color consistency of an image], Perspektivy razvitiya informatsionnykh tekhnologiy [Prospects
for the development of information technologies], 2015, No. 27, pp. 6-17.
10. Silakov N.V., Tassov K.L. Obzor algoritmov obnaruzheniya tekstovykh oblastey na kadrakh
videopotoka [Review of algorithms for detecting text areas on video stream frames], Vestnik
RGGU. Seriya: Informatika. Informatsionnaya bezopasnost'. Matematika [Bulletin of the Russian
State University for the Humanities. Series: Computer Science. Information Security.
Mathematics], 2020, No. 2, pp. 27-45. DOI: 10.28995/2686-679X-2020-2-27-45.
11. Mahajan S., Rani R., Trehan K. DELIGHT-Net: DEep and LIGHTweight network to segment
Indian text at word level from wild scenic images, International Journal of Multimedia Information
Retrieval, 2023, No. 12 (2), pp. 29-37.
12. Gonsales R., Vuds R. TSifrovaya obrabotka izobrazheniy [Digital image processing]. Moscow:
Tekhnosfera, 2012, 1104 p.
13. Shapiro L., Stokman Dzh. Komp'yuternoe zrenie [Computer vision]. Moscow: BINOM, 2013,
752 p.
14. Khomonenko A.D., Yakovlev E.L. Obosnovanie arkhitektury svertochnoy neyronnoy seti dlya
avtonomnogo raspoznavaniya ob"ektov na izobrazheniyakh bortovoy vychislitel'noy sistemoy
[Justification of the architecture of a convolutional neural network for autonomous recognition
of objects in images by an on-board computer system], Naukoemkie tekhnologii v
kosmicheskikh issledovaniyakh Zemli [Science-intensive technologies in space research of the
Earth], 2018, Vol. 10, No. 6, pp. 86-93.
15. Busta M., Neumann L., Matas J. Deep TextSpotter: An End-to-End Trainable Scene Text Localization
and Recognition Framework, 2017 IEEE International Conference on Computer Vision
(ICCV), 2017, pp. 2223-2231.
16. Liu Y., Wang Y., Shi H. A Convolutional Recurrent Neural-Network-Based Machine Learning
for Scene Text Recognition Application. Symmetry, Symmetry, 2023, No. 15 (4), 849.
17. Liu X., Liang D., Yan S., Chen D., Qiao Y., Yan J. FOTS: Fast Oriented Text Spotting with a
Unified Network, Computer Vision and Pattern Recognition, 2018, pp. 5676-5685.
18. Chaitra Y., Dinesh R., Gopalakrishna M., Prakash B.A. Deep-CNNTL: Text Localization from
Natural Scene Images Using Deep Convolution Neural Network with Transfer Learning, Arabian
Journal for Science and Engineering, 2021, No. 47 (9), pp. 9629-9640.
19. He T., Tian Z., Huang W., Shen C., Qiao Y., Sun C. An End-to-End TextSpotter with Explicit
Alignment and Attention, IEEE Conference on Computer Vision and Pattern Recognition,
2018, pp. 5020-5029.
20. Bulatov K.B. A method to reduce errors of string recognition based on combination of several
recognition results with per-character alternatives, Vestnik Yuzhno-Ural'skogo gosudarstvennogo
universiteta. Seriya: Matematicheskoe modelirovanie i programmirovanie [Bulletin
of the South Ural State University. Series: Mathematical modeling and programming], 2019,
Vol. 12, No. 3, pp. 74-88. DOI: 10.14529/mmp190307.
21. Panishchev V.S., Trufanov M.I., Dobroserdov O.G., Khomyakov O.O. Raspoznavanie simvol'noy
informatsii dlya avtomatizatsii proizvodstvennykh protsessov [Recognition of symbolic information
for automation of production processes], Izvestiya Yugo-Zapadnogo gosudarstvennogo universiteta
[News of the South-Western State University], 2021, No. 1, pp. 122-137.
Published
2023-10-23
Issue
Section
SECTION I. INFORMATION PROCESSING ALGORITHMS
