ESTIMATING THE EFFECTIVENESS OF THE METHOD FOR SEARCHING THE ASSOCIATIVE RULES FOR THE TASKS OF PROCESSING BIG DATA

Abstract

The modern databases have significant volume and consist of large masses of information. One of the popular methods of knowledge identification in terms of tasks of analysis and processing of large data volumes is composed of the algorithms for searching the associative rules. The paper solves the problem of building the bases of associative rules for the analysis of the unstructured large data volumes on the basis of searching different regularities considering the importance of their characteristics. The authors propose the method for synthesizing the bases and building the transaction database to calculate the threshold values of support and application of criteria of estimating implicit associations. This allows us to extract repeated and implicit associative rules. To improve the computational effectiveness of extracting the associative rules, the paper applies the genetic algorithm for optimization of input parameters of the characteristic searching space. The developed method shortens the time of rules extraction, reduces the number of generated common rules, and avoid the resource-consuming procedure of pre-processing the synthesized rule base. The authors developed the program and algorithmic module to carry out the experimental research of the proposed method for synthesizing the associative rules on the basis of filtering the input parameters of the search model for solving the tasks of processing the unstructured data. The experiments conducted on the test transaction bases allow us to clarify the theoretical estimations of time complexity of the proposed method that used the genetic algorithm to calculate the weighed support of the set of rules considering the assessment of a priori informative content of the characteristics included in the dataset. The time complexity of the developed method is estimated as  О(I2). The comparative analysis is performed using the test data of the Retail Data with the algorithms Apriori and Frequent Pattern-Growth. The results have proven the effectiveness of the search method on big sets of transactions. The method allows us to reduce the cardinal of an irredundant set of extracted associative rules in more than 40% in comparison with the popular algorithms. The experiments have shown that the method can be effective for the tasks of knowledge discovery in terms of processing large volumes of data.