TWO-STAGE BOOSTING OF BINARY CLASSIFICATION BASED ON THE APPLICATION OF BIOINSPIRED ALGORITHMS
Abstract
In the process of solving a wide range of applied problems, it becomes necessary to decompose
objects. As a result, the classification problem is an urgent problem in modern data mining
systems. Binary classification is one of the most important tasks, and has a number of unsolved
problems. One such problem is the effectiveness of automated classification. In the tasks of automated
classification, it is relevant to use the algorithmic apparatus of evolutionary computing.
Thus, it is advisable to use genetic and bio-inspired algorithms in the task of finding the optimalvalues of the classifier parameters. To solve this problem, it is proposed to apply the particle
swarm algorithm (PSO). This algorithm in the context of the task of finding suboptimal values of
the parameters of the classifier is able to provide high quality classification. A modification of the
algorithm is a dynamic change in the coordinate values that are responsible for the type of kernel
function. This revision can significantly reduce the time spent developing the classifier. To increase
the classification efficiency, it is advisable to use ensembles of algorithms. The paper presents
the structure of a two-level classifier. At the first level of this classifier, an ensemble of simple
classifiers is formed that form the training set, which is further used by the particle swarm
algorithm in the second stage. This approach can significantly reduce time costs, as well as improve
the quality of the resulting solutions. The particle swarm algorithm (PSO), in the context of
the task of finding suboptimal values of the parameters of the classifier, is able to provide high
quality classification. The proposed two-level algorithm has been experimentally tested. A comparison
is made with analogues, comparative charts are given. The described studies show that
the work is of high theoretical significance, and the conducted experimental studies prove high
practical significance.
References
algoritmov [Review of modern research on the problem of the quality of learning algorithms],
Tavricheskiy vestnik informatiki i matematiki [Tauride Bulletin of Informatics and mathematics],
2004, No. 1, pp. 5-24.
2. Kureychik V.M., Kureychik V.V., Rodzin S.I., Gladkov L.A. Osnovy teorii evolyutsionnykh
vychisleniy [Fundamentals of the theory of evolutionary computing]. Rostov-on-Don: YuFU, 2010.
3. Karpenko A.P. Populyarnye algoritmy global'noy poiskovoy optimizatsii. Obzor novykh i
maloizvestnykh algoritmov [Popular algorithms for global search engine optimization. Review
of new and little-known algorithms], Informatsionnye tekhnologii [Information technologies],
2012, No. 7 (Appendix), pp. 1-32.
4. Rodzin S.I., Kureychik V.V. Sostoyanie, problemy i perspektivy razvitiya bioevristik [State, problems
and prospects for the development of bioheuristics], Programmnye sistemy i vychislitel'nye
metody [Software systems and computational methods], 2016, No. 2, pp. 158-172.
5. Kureychik V.M., Zaporozhets D.Yu. Roevoy algoritm v zadachakh optimizatsii [Swarm algorithm
in optimization problems], Izvestiya YuFU. Tekhnicheskie nauki [Izvestiya SFedU. Engineering
Sciences], 2010, No. 7 (108), pp. 28-32.
6. Gladkov L.A., Kureychik V.V., Kureychik V.M. Geneticheskie algoritmy [Genetic algorithms].
Moscow: Fizmatlit, 2010, 368 p.
7. Karpenko A.P. Sovremennye algoritmy poiskovoy optimizatsii. Algoritmy, vdokhnovlennye
prirodoy: ucheb. posobie [Modern search optimization algorithms. Algorithms inspired by nature:
tutorial]. Moscow: Izd-vo MGTU im. N.E. Baumana, 2014, 446 p.
8. Clarke, B., Fokoue E., Zhang H.H. Principles and Theory for Data Mining and Machine
Learning. Springer Science, LLC, 2009, 781 p.
9. Graf H.P., Cosatto E., Bottou L., Durdanovic I., Vapnik V. Parallel Support Vector Machines:
The Cascade SVM, Advances in Neural Information Processing Systems, 2004, Vol. 17,
pp. 521-528.
10. Priyadarshini A., Agarwal S. A Map Reduce based Support Vector Machine for Big Data
Classification, International Journal of Database Theory and Application, 2015, Vol. 8, No. 5,
pp. 77-98.
11. Demidova L., Sokolova Yu. A Novel SVM-kNN Technique for Data Classification, 6-th Mediterranean
Conference on Embedded Computing (MECO’ 2017), 2017, pp. 459-462.
12. Zhang H., Berg A.C., Maire M., Malik J. SVM-KNN: Discriminative Nearest Neighbor Classification
for Visual Category Recognition, Proceedings, 2006 IEEE Computer Society Conference
on Computer Vision and Pattern Recognition, 2016, Vol. 2, pp. 2126-2136.
13. Vorontsov K.V. Kombinatornye otsenki kachestva obucheniya po pretsedentam [Combinatorial
evaluations of the quality of training based on precedents], Dokl. RAN [Reports of the Russian
Academy of Sciences], 2004, Vol. 394, No. 2, pp. 175-178.
14. D'yakonov A.G. Analiz dannykh, obuchenie po pretsedentam, logicheskie igry, sistemy
WEKA, RapidMiner i MatLab (Praktikum na EVM kafedry matematicheskikh metodov
prognozirovaniya): ucheb. posobie [Data analysis, case studies, logic games, WEKA,
RapidMiner and MatLab systems (computer Workshop of the Department of mathematical
forecasting methods): textbook]. Moscow: Izdatel'skiy otdel fakul'teta VMK MGU im.
M.V. Lomonosova, 2010, 278 p.
15. Lean Yu, Shouyang Wang, Kin Keung Lai, Ligang Zhou. Bio-Inspired Credit Risk Analysis.
Computational Intelligence with Support Vector Machines. Springer-Verlag Berlin Heidelberg,
2008, 244 p.
16. V'yugin V.V. Matematicheskie metody teorii mashinnogo obucheniya i prognozirovaniya:
ucheb. posobie [Mathematical methods of the theory of machine learning and forecasting:
textbook]. Moscow: MFTI, 2013, 379 p.
17. Demidova L.A., Sokolova Yu.S. Aspekty primeneniya algoritma roya chastits v zadache
razrabotki SVM-klassifikatora [Aspects of applying the particle swarm algorithm to the problem
of developing an SVM classifier], Vestnik Ryazanskogo gosudarstvennogo
radiotekhnicheskogo universiteta [Bulletin of the Ryazan state radio engineering University],
2015, No. 3 (53), pp. 84-92.
18. Duggal P.S., Paul S., Tiwari P. Analytics for the Quality of Fertility Data using Particle
Swarm Optimization, International Journal of Bio-Science and Bio-Technology, 2015, Vol. 7,
No. 1, pp. 39-50.
19. Kureychik V.V., Kureychik V.M., Sorokoletov P.V. Analiz i obzor modeley evolyutsii [Analysis
and review of evolution models], Izvestiya Rossiyskoy akademii nauk. Teoriya i sistemy
upravleniya [Proceedings of the Russian Academy of Sciences. Theory and control systems],
2007, No. 5, pp. 114-126.
20. Chapelle O., Vapnik V., Bousquet O., Mukherjee S. Choosing Multiple Parameters for Support
Vector Machine, Machine Learning, 2002, Vol. 46, pp. 131-159.
