INTEGRATION OF SEGMENTATION, TRACKING AND CLASSIFICATION MODELS TO SOLVE VIDEO ANALYTICS PROBLEMS

Abstract

The integration of several models into one technical vision system will allow solving more complex tasks. In particular, for mobile robotics and unmanned aerial vehicles (UAVs), the lack of data sets for various conditions is an urgent problem. In the work, the integration of several models is proposed as a solution to this problem: segmentation, maintenance and classification. The segmentation model allows you to select arbitrary objects from frames, which allows it to be used in nondeterministic and dynamic environments. The classification model allows you to determine the objects necessary for navigation or other use, which are then accompanied by a third model. The paper describes an algorithm for model aggregation. In addition to models, the key element is the correction of model predictions, which allows you to segment and accompany various objects reliably enough. The procedure for correcting model predictions solves the following tasks: adding new objects to accompany, validating segmented object masks and clarifying the associated masks. The versatility of this solution is confirmed by working in difficult conditions, for example, underwater photography or images from UAVs. An experimental study of each of the models was carried out in an open area and indoors. The data sets used make it possible to assess the applicability of models for mobile robotics tasks, that is, to identify possible obstacles in the robot's path, for example, a curb, as well as moving objects such as a person or a car. They demonstrated a sufficiently high quality of work. For most classes, the indicators exceeded 80% by various metrics. The main errors are related to the size of the objects. The conducted experiments clearly demonstrate the versatility of this solution without additional training of models. Additionally, a study of performance on a personal computer with various input parameters and resolution was conducted. Increasing the number of models significantly increases the computational load and does not reach real time. Therefore, one of the directions of further research is to increase the speed of the system