Search

Search Results

• ALTERNATIVE APPROACHES TO NLP MODEL SCALE-UP: AN ANALYSIS OF APPROACHES TO OPTIMIZING DATA AND COMPUTATION VOLUME WHEN TRAINING LARGE-SCALE LANGUAGE MODELS

  К.I. Ralko , N. Е. Sergeev

  152-172

  2026-07-07

  Abstract ▼

  This paper focuses on overcoming the systemic limitations of the large-scale language model (LLM) scaling paradigm, which are related to data exhaustion and exponential growth in computational costs. This enables the development of more efficient approaches to building NLP models without sacrificing their performance. The goal of this study is to compare the performance of a standard transformer architecture (nanoGPT) and a model using semantic embeddings (nanoSonar) for language modeling tasks under resource constraints. Working with conceptual embeddings allows us to identify deeper linguistic patterns and reduce the amount of required training data, significantly improving modeling efficiency. The study utilized the TinyStories dataset, which includes short narratives with a clear structure. Before implementing the models, the data was preprocessed: for nanoGPT, tokenization was performed using the BPE method, and for nanoSonar, text was converted into semantic embeddings using a pretrained Sonar model. The models were evaluated using the loss and perplexity metrics. The results showed that the nanoSonar model provides significantly lower perplexity (6.609 versus 39.151 for nanoGPT) and demonstrates more robust training dynamics at later stages. This paper presents an analysis of modern approaches to scaling optimization (MoE, distillation, PEFT) and promising architectures (LRM, SSM, RWKV), and provides practical recommendations for applying models operating in the space of semantic embeddings to domain-specific problems and systems with limited computational resources. The results of this study can be useful in developing efficient language models that combine high generation quality with a cost-effective architecture.

• A STOCHASTIC FRAMEWORK FOR MODELING TRADERS’ COGNITIVE RISK UNDER VOLATILITY IN DECENTRALIZED FINANCIAL MARKETS

  D. G. Veselova , N. Е. Sergeev

  189-199

  2025-12-30

  Abstract ▼

  This study is devoted to the development of a stochastic model of traders’ cognitive risk as a core component of an intelligent decision support system (DSS) for decentralized cryptocurrency markets.
  The relevance of the research is determined by the specific characteristics of the DeFi environment, which include high and nonstationary volatility, the absence of centralized stabilization mechanisms, information asymmetry, and a strong influence of behavioral factors on trading decisions. Under these conditions, traditional deterministic and static DSS frameworks demonstrate limited effectiveness, as they fail to account for the dynamic perception of risk by market participants and the associated cognitive biases.
  The objective of this research is to formalize traders’ cognitive risk as a memory-dependent stochastic process and to integrate the proposed model into the architecture of an adaptive DSS for risk management. To achieve this objective, a stochastic differential equation is developed to describe the dynamics of cognitive risk as a function of market volatility and prevailing market regimes. In addition, a probabilistic transition kernel is introduced to link objective market characteristics with the subjective perception of risk. For parameter estimation, an identification framework based on the Expectation–Maximization algorithm combined with particle filtering is proposed, enabling robust inference in the presence of nonlinear dynamics and latent state variables. The research methodology includes numerical simulations on synthetic data, parameter estimation using real cryptocurrency time series, and validation of the proposed approach through walk-forward and purged K-fold schemes. The quality of probabilistic forecasts is evaluated using the Negative Log-Likelihood (NLL), Brier Score, and Expected Calibration Error (ECE) metrics. Experimental results demonstrate that incorporating the stochastic cognitive layer improves probabilistic forecasting performance by an average of 10–15%, reduces NLL by approximately 8%, decreases the Brier Score by about 11%, and lowers ECE by nearly 35%. Furthermore, the accuracy of predicting key transitions between market regimes increases by 5–7 percentage points. The obtained results confirm the effectiveness of the proposed stochastic cognitive-risk model and demonstrate its applicability for the development of adaptive DSS solutions in the DeFi domain. The proposed framework provides a foundation for further research on predictive models of trader behavior and the design of intelligent risk-management systems for decentralized financial ecosystems.

• FORMALIZATION OF A SET OF INFORMATIVE SIGNS THE DYNAMICS OF MANIPULATION BY CONTROL DEVICES TO SOLVING THE PROBLEM OF DIAGNOSING THE PRODUCTIVITY OF BTS OPERATORS

  A.V. Skrinnikova, N. E. Sergeev

  2021-01-19

  Abstract ▼

  Informative signs of the dynamics of manipulation by control devices such as a mouse and
  keyboard play an important role in the development of software complexes for the identification of
  biotechnical systems (BTS) operators by their individual dynamics, in solving problems of diagnostics
  of various psycho emotional states and operator productivity. It finds application in the
  spheres of technical and law enforcement security, medical and energy spheres, etc. The purpose
  of this work is to formalize a set of informative signs of the dynamics of manipulation by control
  devices to solving the problem of diagnosing the productivity of BTS operators. To achieve this
  goal, an overview of the most frequently used features is presented, the Bayesian approach is considered
  in the statistical formulation of the recognition problem, a set of informative sings of the
  dynamics of keyboard handwriting and the dynamics of mouse manipulations is formalized basedon the results of a number of works. Operator productivity forecast based on fuzzy rules based on
  selected criteria gave an accuracy of more than 90%. The advantage of using the dynamics of
  manipulation of the control devices of the BTS operators is the absence of special equipment that
  requires additional costs.

• ALGORITHM FOR PRE-PROCESSING VIDEO IMAGES TO INCREASE THE ACCURACY OF SMALL OBJECT DETECTION

  V.V. Kovalev, N.E. Sergeev

  2021-12-24

  Abstract ▼

  Recognition of certain patterns in video images captured by a camera is carried out using
  training methods based on convolutional neural networks. The larger the number of images with
  multiple features and the more diverse the training sample of video images, the better the convolutional
  neural networks extract features from the sequence of video images that were not included in
  the training sample. This is a consequence of increasing the accuracy of detecting visual images on
  video images containing features of target images. However, there are limitations in improving the
  detection performance when the size of the image to be detected is much smaller than the background
  area, or when the image is described with little information. To solve problems of this kind, the authors
  of the article have developed an algorithm for the spatio-temporal integration of information
  about the movement of dynamic images. The algorithm processes a fixed number of video images at
  certain points in time and extracts new independent signs of motion of dynamic images based on
  space-time processing of video images. Further, it combines new local motion features with the original
  video image features. This allows you to add a motion feature of dynamic images while preserving
  the original image features that describe static images. Areas of the video image that characterize
  the motion feature are displayed in a «color» cluster. The use of pre-processing is aimed at improving the accuracy of pattern detection, provided there are dynamic visual images on a static background.
  If the camera is in scan mode, a static background can be provided with a video stabilizer.
  Experimentally, estimates of integral criteria for the accuracy of detection neural network algorithms
  have been obtained, showing an increase in the accuracy of detecting visual images using
  the algorithm for spatial-temporal integration of motion information.

• ACCELERATION OF THE DIRECT PASSAGE IN THE IMPLEMENTATION OF CNN ON A LIMITED COMPUTING RESOURCE

  А.Е. Shchelkunov, V.V. Kovalev, I. V. Sidko, N. Е. Sergeev

  2022-04-21

  Abstract ▼

  The work is devoted to the optimization of the neural network architecture for its launch on
  a limited computing resource. Several optimization approaches are considered, estimates of the
  complexity and execution time of the forward pass of the neural network are given. Comparative
  estimates of the complexity of the network using different optimization approaches are given.
  The paper presents an analysis of the selected network architecture, and estimates of the computational
  complexity of individual components (modules) of the architecture are obtained. An analysis
  of possible optimization methods for each module was made. The parameters of the considered
  modules, the sizes of the input and output tensors are described. Several architectures were tested
  to optimize the feature extraction module, ResNet 50, ResNet 18, MobileNet v3 small, MobileNet
  v3 large. A comparative analysis of the computational complexity and execution time of the forward
  pass for each architecture is presented. Forward pass times were measured on Nvidia's
  Jetson AGX Xaver embedded computing device. Estimates of the execution time of the direct pass
  for each module of the considered neural networks are presented. The paper presents the results of
  comparing neural network accuracy estimates before and after architecture optimization. The test
  data set consists of 100 video recordings. 5 different typical objects are involved in test videos,
  10 different scenarios are recorded for each object class. For each of the developed architectures,
  accuracy estimates were obtained, and a comparative analysis was made.

• DEVELOPMENT AND RESEARCH OF A QUANTUM GRAPH MODEL FOR IMAGE COMPRESSION AND RECONSTRUCTION

  А.N. Samoilov, S.М. Gushanskiy, N.Е. Sergeev, V.S. Potapov

  2024-11-10

  Abstract ▼

  The article discusses in detail the methods and approaches to the application of quantum algorithms
  for solving optimization and image processing problems. Particular attention is paid to quantum approximate
  optimization (QAO) and the use of quantum networks for data compression and reconstruction problems.
  QAO is a hybrid algorithm that combines quantum and classical computational processes, allowing
  one to efficiently solve complex combinatorial problems. QAO is based on parameterized unitary operations
  that are optimized during iterations. This approach makes it possible to consider the unique features
  of the quantum nature of information, which in some cases allows achieving higher performance than
  when using exclusively classical methods. In the process of implementing QAO, one of the main obstacles
  remains the problem of noise, which can arise, for example, when using CNOT gates. The article discusses
  various strategies for reducing the noise level, which is an important task for ensuring the stability and
  improving the accuracy of quantum algorithms. For example, methods for isolating individual operations
  and correcting errors are considered, which allows one to minimize the impact of noise on the calculation
  results and improve the accuracy of quantum optimization. The authors also propose a graph interpretation
  of quantum models based on the use of tensor networks. This approach allows for efficient simplification
  of computational graphs, thereby optimizing the resources required to perform complex quantum
  operations. This method also demonstrates high efficiency in image compression and restoration tasks,
  which opens up new prospects for the application of quantum networks in data processing. The article
  describes the structure of quantum networks, including multilayer quantum gates, which allow for deeper
  and more detailed image processing, providing both efficient compression and high-quality data restoration.
  An analysis of various types of quantum gates, such as Hadamard, Pauli-X, Pauli-Y, and T-gates,
  was also conducted. These gates play a key role in the efficiency of quantum algorithms, since each of
  them contributes to quantum dynamics and the way quantum states are manipulated

• FORECASTING ELECTRICITY CONSUMPTION BY INDUSTRIAL ENTERPRISES (REVIEW)

  I.V. Emanov, N.Е. Sergeev

  2024-10-08

  Abstract ▼

  Large consumers of electricity mainly purchase electricity on the wholesale electricity and capacity
  market, for example, industrial enterprises of ferrous metallurgy. For the production of products, large
  industrial enterprises daily order hourly volumes of electricity consumption for two days in advance, if
  necessary, enterprises have the right to send adjusted values for the day preceding the day of consumption.
  At the same time, for deviations from the planned hourly volumes, enterprises incur additional costs,
  which are included in the electricity tariff. One of the most important factors that affect the forecasting of
  hourly electricity consumption are: the variety of types of main and auxiliary equipment, the capacities of
  electricity consumers carrying out the technological processes of the enterprise; the intensity of production
  load and operating modes depending on the production of the product range; the frequent use of
  hours of maximum electric power during the Days; energy-intensive production. To build forecast data for
  time series, a model is built to predict hourly electricity consumption by an industrial enterprise and has a
  large number of input data that have a probabilistic component. Consideration of various methods for
  forecasting time series of electricity consumption of industrial enterprises seems to be an urgent scientific
  and technical task. This is due to the requirements of minimization, firstly, of jumps and failures in the
  operation of generating capacities of the energy system of the region in which the enterprise is located
  (since the load, for example, of ferrous metallurgy enterprises can reach up to 10% of the total consumption
  of the region), and secondly, additional costs associated with the purchase/sale of volumes of electricity
  consumed in excess of the application/unused in case of inaccurate planning of hourly volumes of electricity
  consumed, which are included in the electricity tariff.

• PREDICTIVE ANALYTICS FOR DECISION-MAKING IN DECENTRALIZED SYSTEMS

  N.Y. Sergeev, D.G. Veselova

  2024-05-28

  Abstract ▼

  Currently, the relevance of using crypto assets is growing rapidly. In recent years, cryptocurrency
  trading has become one of the most discussed topics in the world of finance and investment.
  Cryptocurrencies such as Bitcoin, Ethereum, attract the attention of millions of people due to their innovativeness,
  high profit potential, and decentralization possibilities. The blockchain technology, on which
  cryptocurrencies are based, is one of the most innovative and promising technologies in the market. Studying
  cryptocurrency trading helps understand how private investors and companies can use blockchain
  technologies for investment and business development. One of the main reasons for the popularity of
  cryptocurrency trading is its high level of volatility. The cryptocurrency exchange rate can change quickly,
  providing opportunities for profit. This article focuses on exploring the use of predictive analytics for
  decision-making in decentralized systems using cryptocurrency trading on centralized and decentralized
  exchanges as an example. The research conducted in this work aims to investigate decentralized and centralized
  systems to further develop decision support systems. A general description and operation schemes
  of decentralized and centralized dynamic systems are provided using cryptocurrency exchanges as a research
  example. This scientific article examines the typical structure of centralized and decentralized
  cryptocurrency exchanges, analyzing the fundamental components and principles of their functioning.
  The article discusses the internal organization of the exchange, including the system for storing digital assets, transaction execution mechanisms, security provisions, and risk management. It also examines the
  interaction between the exchange and market participants, as well as regulatory bodies. Furthermore, this
  scientific article explores the rules and principles of operation for traders and market makers on centralized
  and decentralized cryptocurrency exchanges. It covers the main strategies and tactics used by market
  participants to ensure liquidity and optimize trading operations. The article compares the trading approaches
  on different types of cryptocurrency exchanges considering their specific features and impact on
  cryptocurrency price dynamics. The presented results can contribute to a deeper understanding of
  cryptocurrency trading processes and optimize decision-making strategies for investors and traders in the
  crypto asset market.

• EXPANSION OF THE FEATURE SPACE IN THE TASK OF SMALL OBJECT DETECTION IN IMAGES

  V.V. Kovalev, N.E. Sergeev

  2024-04-16

  Abstract ▼

  One of the current trends in creating early object detection systems is the development of algorithms
  for searching and recognizing small objects in images. In the early detection task, it is necessary
  to recognize objects at long distances from the place where they were recorded by the camera.
  The image in the image of such objects is represented by a small compact group of pixels, which
  undergoes spatial and brightness changes from frame to frame. To successfully solve this problem,
  real-world target objects must have large physical dimensions. In addition to the physical dimensions
  of the object, the image of the object in the image is influenced by a large number of factors: the resolution
  of the camera matrix, the focal length of the lens, the photosensitivity of the matrix, etc. The
  vector for solving this problem is directed towards convolutional neural networks. However, even
  advanced convolutional neural network architectures face challenges in finding and recognizing
  small objects in images. This problem is directly related to the effect of overtraining the neural network
  model. Retraining of a neural network model can be assessed based on learning curve analysis.
  To reduce the likelihood of overfitting, special methods are used, which are united by the term regularization.
  However, in recognizing small-sized objects, existing regularization methods are not
  enough. The work examines the developed algorithm for preprocessing a sequence of video frames,
  which increases the original feature space with a new independent feature of movement in the frame.
  The preprocessing algorithm is based on spatiotemporal filtering of a sequence of video frames, the
  application of which extends to a wide range of convolutional neural network architectures. To study
  the characteristics of accuracy and recognition of convolutional neural networks, datasets of
  grayscale images and images with a sign of motion were generated based on the 3D graphics development
  environment Unreal Engine 5. The work presents a criterion for the small size of objects in
  images. The accuracy characteristics of the test model of the convolutional neural network were
  trained and assessed, and the dynamics of the learning curves of the test model were analyzed. The
  positive influence of the proposed algorithm for preprocessing a sequence of video frames on the
  integral accuracy of detection of small-sized objects is shown.

• IMPLEMENTATION OF CONVENTIONAL NEURAL NETWORKS ON EMBEDDED DEVICES WITH A LIMITED COMPUTING RESOURCE

  V.V. Kovalev, N.E. Sergeev

  2022-01-31

  Abstract ▼

  Large amounts of video data captured by sensor sensors in various spectral ranges, the significant
  size of convolutional neural network architectures create problems with the implementation of
  neural network algorithms on peripheral devices due to significant limitations of computing resources
  on embedded computing devices. The article discusses the use of algorithms for automatic search and
  pattern recognition based on machine learning methods, implemented on embedded devices with a
  computing resource Graphics Processing Unit. Detection convolutional neural networks «You Only
  Look Once V3» and «You Only Look Once V3-Tiny» are used as a search and pattern recognition algorithm,
  which are implemented on embedded computing devices of the NVIDIA Jetson line, located in
  different price ranges and with different computing resources ... Also, in the work, the estimates ofalgorithms on embedded devices are experimentally calculated for such indicators as power consumption,
  forward passage time of a convolutional neural network, and detection accuracy.
  On the basis of solutions implemented, both at the hardware level and in software, presented by
  NVIDIA, it becomes possible to use deep neural network algorithms based on the convolution
  operation in real time. Computational optimization methods offered by NVIDIA are considered.
  Experimental studies of the influence of computations with reduced accuracy on the speed and
  accuracy of object detection in images of the investigated architectures of convolutional neural
  networks, which were previously trained on a sample of images consisting of the PASCAL VOC
  2007 and PASCAL VOC 2012 datasets, have been carried out.