SUBSYSTEM FOR AUTOMATIC TEXT ANNOTATION BASED ON MACHINE LEARNING METHODS

Abstract

This paper considers the problem of automatic text annotation. The formulation of the problem is considered. The relevance and importance of developing effective methods and software systems for solving the problem of automatic text summarization in modern information systems is substantiated. Definitions of the concepts “data” and knowledge are given.” A list of tasks related to the Data Mining direction is described. The Text Mining problem and existing methods for solving it are described in detail. The problem of summarizing texts is considered. The main stages of solving the summation problem are highlighted. The main methods of automatic text processing are described, their advantages and disadvantages are highlighted. Abstractive summarization and extractive summarization methods are discussed in detail. A comparative analysis of the effectiveness of various abstracting and quasi-abstracting methods has been carried out, their key advantages and disadvantages have been highlighted. A brief description of the encoder-decoder architecture is given from the point of view of using this architecture in the developed algorithm for automatic text summarization. A description of the model of recurrent neural networks is given, the advantages and disadvantages of such models are noted. The architecture of a recurrent neural network is considered in relation to solving the problem of automatic text summarization. A description of the modified model of a recurrent neural network – a neural network with long short-term memory – is given. A description of the proposed automatic abstracting algorithm and the settings of its main parameters are given. A description of the developed automatic abstracting software subsystem is given. Computer modeling is performed and the results obtained during computational experiments are presented. The quality of the solutions obtained was assessed. The optimal parameters of the developed software system are determined. Directions for continuing research are formulated.