THE COMMON ALTERNATIVE APPROACH FOR THE EFFICIENT DEEP LEARNING NEURAL NETWORKS

Abstract

This paper proposes a new approach to the organization of computational structures of layers
and inter-layer connections in the construction of artificial neural networks for solving a wide
range of problems of multidimensional data processing. The main problem of building deep learning
networks is the necessity of introducing a large number of network training parameters. Available
working instances of such networks contain billions of parameters, which allows to achieve
high efficiency of such networks. The downside of such a widely used structure of networks in the
form of multilayer sieve structures is the high cost of training networks with a large number of
structurally similar convolution layers by the back-propagation method. A solution to the problem
of increasing the efficiency of such multilayer structures can be found in the use of hybrid layers
realizing data granularity operations, which were developed in our work. The new hybrid models
use matrix information elements instead of vector values of training parameters, which allow encoding
subsets of data values (information granules) instead of encoding individual data points as
in classical convolutional networks. The proposed hybrid layers are trained without a teacher and
allow parallel implementation of learning algorithms, which is fundamentally different from sequential
backpropagation algorithms as a result, the computational efficiency of similar hybrid
neural networks can be significantly increased. The theoretical approach to modeling and optimizing
the structures of deep learning networks proposed in this paper can be extended to a wide
range of computational intelligence problems.