Search
Search Results
-
IMPROVING THE QUALITY OF MULTI-MODAL DATA BASED ON THE HYBRIDIZATION OF PREPROCESSING AND MULTISENSORY FUSION METHODS
А. А. Aleksandrov , М.А. Butakova21-382026-09-10Abstract ▼The article explores the methods of data preprocessing and merging of multi-modal data. The data collected from multi-sensor devices contains noise, anomalies, and includes sensor failures. The use of raw data inevitably leads to false patterns when using machine learning models in which this data is used. To improve the quality of data, it is necessary to use special methods for preliminary processing and merging. The aim of the work is to create a hybrid preprocessing and multisensory data fusion controller operating at the peripheral computing level. To achieve this, the tasks of removing the consequences of technical failures and noise from the data, time synchronization, scaling of features and combining duplicate sources, considering their dynamic weight, have been solved. The following methods were applied: inter-quartile scale to detect outliers, linear interpolation to replace them, high-pass filter to suppress noise, and weighed multisensory fusion with dynamic weight calculation. In addition, methods for processing visual, sensory, and acoustic data within a hybrid pipeline are presented that can be adapted for implementation at the peripheral computing level. The experimental verification of the developed pipeline was carried out on a simulated set of climatic data with artificially introduced noises and anomalies.
The proposed approach has demonstrated high efficiency of data recovery. For air temperature, the coefficient of determination (R2) increased from 0.9807 to 0.9964 with a slight change in the mean absolute percentage error (MAPE) from 10.11% to 10.48%. For relative humidity, the R2 metric increased from 0.9248 to 0.9675, and the MAPE error decreased from 2.87% to 2.57%. The most significant improvement has been achieved for atmospheric pressure. The R2 metric increased from a negative value of -0.124 to 0.974, and the MAPE error decreased from 0.12% to 0.06%. The results confirm that the proposed pipeline reduces the error and improves the quality of multisensory data, minimizes the amount of data being transferred. The practical value lies in creating a continuous cycle of local filtering and merging of data without accumulation of errors. -
METHODS AND ALGORITHMS FOR TEXT DATA CLUSTERING (REVIEW)
V.V. Bova, Y.A. Kravchenko, S.I. Rodzin2022-11-01Abstract ▼The article deals with one of the important tasks of artificial intelligence – machine processing
of natural language. The solution of this problem based on cluster analysis makes it possible
to identify, formalize and integrate large amounts of linguistic expert information under conditions
of information uncertainty and weak structure of the original text resources obtained from
various subject areas. Cluster analysis is a powerful tool for exploratory analysis of text data,
which allows for an objective classification of any objects that are characterized by a number of
features and have hidden patterns. A review and analysis of modern modified algorithms for agglomerative
clustering CURE, ROCK, CHAMELEON, non-hierarchical clustering PAM, CLARA
and the affine transformation algorithm used at various stages of text data clustering, the effectiveness
of which is verified by experimental studies, is carried out. The paper substantiates the
requirements for choosing the most efficient clustering method for solving the problem of increasing the efficiency of intellectual processing of linguistic expert information. Also, the paper considers
methods for visualizing clustering results for interpreting the cluster structure and dependencies
on a set of text data elements and graphical means of their presentation in the form of
dendograms, scatterplots, VOS similarity diagrams, and intensity maps. To compare the quality of
the algorithms, internal and external performance metrics were used: "V-measure", "Adjusted
Rand index", "Silhouette". Based on the experiments, it was found that it is necessary to use a
hybrid approach, in which, for the initial selection of the number of clusters and the distribution of
their centers, use a hierarchical approach based on sequential combining and averaging the characteristics
of the closest data of a limited sample, when it is not possible to put forward a hypothesis
about the initial number of clusters. Next, connect iterative clustering algorithms that provide
high stability with respect to noise features and the presence of outliers. Hybridization increases
the efficiency of clustering algorithms. The research results showed that in order to increase the
computational efficiency and overcome the sensitivity when initializing the parameters of clustering
algorithms, it is necessary to use metaheuristic approaches to optimize the parameters of the
learning model and search for a global optimal solution.








