THE COMMON ALTERNATIVE APPROACH FOR THE EFFICIENT DEEP LEARNING NEURAL NETWORKS

Abstract

This paper proposes a new approach to the organization of computational structures of layers and inter-layer connections in the construction of artificial neural networks for solving a wide range of problems of multidimensional data processing. The main problem of building deep learning networks is the necessity of introducing a large number of network training parameters. Available working instances of such networks contain billions of parameters, which allows to achieve high efficiency of such networks. The downside of such a widely used structure of networks in the form of multilayer sieve structures is the high cost of training networks with a large number of structurally similar convolution layers by the back-propagation method. A solution to the problem of increasing the efficiency of such multilayer structures can be found in the use of hybrid layers realizing data granularity operations, which were developed in our work. The new hybrid models use matrix information elements instead of vector values of training parameters, which allow encoding subsets of data values (information granules) instead of encoding individual data points as in classical convolutional networks. The proposed hybrid layers are trained without a teacher and allow parallel implementation of learning algorithms, which is fundamentally different from sequential backpropagation algorithms as a result, the computational efficiency of similar hybrid neural networks can be significantly increased. The theoretical approach to modeling and optimizing the structures of deep learning networks proposed in this paper can be extended to a wide range of computational intelligence problems.