AN INTELLIGENT SYSTEM OF TECHNICAL VISION FOR DETECTING OBSTACLES AND PREDICTING THE BEHAVIOR OF MOVING OBJECTS ON RAILWAY TRACKS
Abstract
Currently, the improvement of the quality of transport and logistics services provided is directly
related to the introduction of new and modernization of existing technologies of
informatization and digitalization. One of the most urgent tasks solved by the introduction of digital
technologies into existing technological processes is to improve the safety of train traffic.
The analysis of domestic and foreign works devoted to the development of train safety improvement
systems has shown that one of the methods of solving the task is the development and implementation
of vision systems for detecting infrastructure objects and obstacles in the course of train
movement. This is especially true when train speeds increase when it is difficult for the driver to
correctly assess the current situation and make an operational decision. This paper describes the
implementation of a vision system for unmanned trains. Within its framework, a new approach to
the training of a highly specialized mask neural network was implemented. The main task of this
system is to recognize obstacles and human figures against the background of the railway infrastructure
determine their location relative to the tracks and assess this situation from the point of
view of traffic safety. To obtain a higher-quality mask, the approach of simultaneous use of images
of standard CVS cameras and cameras with the higher resolution was used. This method is able toimprove the quality of recognition, especially at large distances, when the object of interest is not
noticeable in the complex environment surrounding it. The work performed has shown good results
in identifying objects on railway tracks. The creation of a prototype of such a system and
equipping it with traction rolling stock will allow for the timely detection of obstacles and people
on the train path, which contributes to improving the level of train safety.
References
obstacles on railways based on manifold Kalman filtering, 2018 IEEE 23rd International
Conference on Emerging Technologies and Factory Automation (ETFA). IEEE, 2018, Vol. 1,
pp. 768-775.
2. Minakov V.A., Fomenko V.K. Tekhnologiya mashinnogo zreniya na lokomotivakh dlya
identifikatsii putevykh signalov [Technology of machine vision on locomotives for identification of
track signals], Mir transporta [Mir transport.], 2020, Vol. 17, No. 6, pp. 62-72.
3. Mukojima H. et al. Moving camera background-subtraction for obstacle detection on railway tracks,
2016 IEEE international conference on image processing (ICIP). IEEE, 2016, p. 3967-3971.
4. Kyatsandra A.K. et al. Development of TRINETRA: A Sensor Based Vision Enhancement
System for Obstacle Detection on Railway Tracks, IEEE Sensors Journal, 2022.
5. He D. et al. Obstacle detection in dangerous railway track areas by a convolutional neural
network, Measurement Science and Technology, 2021, Vol. 32, No. 10, pp. 105401.
6. Sheikh Y., Zhai Y., Shafique K. and Shah M. Visual monitoring of railroad grade crossing,
Proc. SPIE Sensors and Command Control Communications and Intelligence (C3I) Technologies
for Homeland Security and Homeland Defense III, 2004, Vol. 5403, pp. 654-660.
7. Konovalenko I. Overview of methods for estimation the observed velocity of the object in the
video stream, Upravlenie, informatsiya i optimizatsiya (VI TMSH) [Management, Information
and optimization (VI TMSH)], 2014, pp. 34-34.
8. Le James. How to do Semantic Segmentation using Deep learning. Available at:
https://nanonets.com/blog/how-to-do-semantic-segmentation-using-deep-learning.
9. Lateef F., Ruichek Y. Survey on semantic segmentation using deep learning techniques,
Neurocomputing, 2019, Vol. 338, pp. 321-348.
10. Vizil'ter Yu.V., Zheltov S.Yu., Bondarenko A.V. i dr. Obrabotka i analiz izobrazheniy v
zadachakh mashinnogo zreniya [Image processing and analysis in machine vision problems].
Moscow: Fizmatkniga, 2010, 672 p.
11. He K. et al. Mask r-cnn, Proceedings of the IEEE international conference on computer vision,
2017, pp. 2961-2969.
12. TensorFlow 2 Detection Model Zoo. Available at: https://github.com/tensorflow/ models/
blob/master/research/object_detection/g3doc/tf2_detection_zoo.md.
13. Gulli A., Kapoor A., Pal S. Deep learning with TensorFlow 2 and Keras: regression,
ConvNets, GANs, RNNs, NLP, and more with TensorFlow 2 and the Keras API. Packt Publishing
Ltd, 2019.
14. Bradski G., Kaehler A. OpenCV, Dr. Dobb’s journal of software tools, 2000, Vol. 3, pp. 2.
15. ROS Kinetic Kame. - URL: http://wiki.ros.org/kinetic.
16. Chang Q., Maruyama T. Real-time stereo vision system: a multi-block matching on GPU,
IEEE Access, 2018, Vol. 6, pp. 42030-42046.
17. Banz C., Blume H., Pirsch P. Real-time semi-global matching disparity estimation on the
GPU, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).
IEEE, 2011, pp. 514-521.
18. Fairchild C., Harman T.L. ROS Robotics By Example: Learning to control wheeled, limbed,
and flying robots using ROS Kinetic Kame. Packt Publishing Ltd, 2017.
19. Karpachevskiy V.V. Pravila tekhnicheskoy ekspluatatsii zheleznykh dorog [Rules of technical
operation of railways], 2017.
20. Sam Schauland, Joerg Velten, Anton Kummert. Motion-Based Object Detection for Automotive
Applications using Multidimensional Wave Digital Filters, VTC Spring 2008 - IEEE Vehicular
Technology Conference, Singapore, Singapore, 20 May 2008, pp. 2700-2704.
