Search

Search Results

• A NEURAL NETWORK-BASED METHOD FOR EXTRACTING CAUSAL RELATIONS USING COMPARATIVE ANALYSIS OF SYNTHETIC AND OPEN CORPORA

  А.N. Tselykh , I. А. Valukhov

  160-169

  2026-09-10

  Abstract ▼

  The automatic extraction of causal relations is critically important for decision support systems, but its development is hindered by the scarcity of annotated corpora. The aim of this study is to comparatively analyze the effectiveness of three types of training data: open expert corpora, a manually annotated political corpus, and synthetic data generated by a large language model. Experiments were conducted using the DistilBERT architecture with token-level BIOES tagging. Synthetic data were generated with the Llama-3 LLM by encapsulating causes and effects in XML markers, followed by deterministic conversion into the BIOES format. For comparison, two expert corpora (EventStoryLine and SemEval-2010 Task 8), the expert-annotated political corpus PolitiCAUSE, and a synthetic sample of 1,100 balanced examples were used. The model trained on synthetic data achieved a Macro F1 score of 0.736, which is 6.7% higher than the result obtained from training on the expert corpora (0.690). Training on the political corpus PolitiCAUSE (24,417 examples) yielded a substantially lower result, with a Macro F1 score of 0.367. This discrepancy is attributed to class imbalance (approximately 15% causal tokens versus 50% in the synthetic sample) and to differences in task formulation: PolitiCAUSE is oriented toward binary sentence-level classification of causal presence, whereas the present study addresses token-level span labeling. The hypothesis that annotation quality and class balance are more important than data volume is confirmed. The proposed LLM-generation pipeline makes it possible to create training datasets that outperform expert corpora without direct manual annotation costs. The method is recommended for rapid deployment of causal relation extraction systems in new domains.

• ALGORITHM FOR FILTERING "HINT INJECTIONS" WHEN USING SPATIAL INFORMATION

  S.L. Belyakov , L.А. Izrailev , О.N. Pokusaev

  32-45

  2026-07-07

  Abstract ▼

  The integration of large language models (LLM) into geographic information systems (GIS) opens up new opportunities for spatial analysis, but it is accompanied by specific vulnerabilities such as "hint injection" (prompt injection). Such attacks allow attackers to bypass LLM security mechanisms, manipulate issuance, gain access to confidential information, and violate data integrity. Using space allows you to access an object not directly, but through its spatial relationships with other objects. Existing keyword or template filtering methods do not provide reliable protection due to the constant emergence of new attack scenarios. This determines the relevance of developing adaptive, self-learning algorithms for filtering queries for industrial injections to large language models. The aim of the study is to develop an algorithm for filtering prompt injections for LLM, based on the Case-Based Reasoning (CBR) method. The paper proposes an algorithm for comparing LLM queries with a database of previously known promt injections. The experiment showed that as the database of use cases accumulates, the accuracy of detecting prompt injections increases from 42% to 83%. At the same time, the processing time for a single request increases slightly (from 0.18 to 0.19 seconds with a 23% increase in the database). Approaches to generalizing the precedent base and introspection of the precedent base were also proposed. The proposed algorithm makes it possible to increase the security of LLM-interface systems against prompt injections due to adaptivity and self-learning. The practical significance lies in the possibility of implementing the developed filter into information systems to prevent leaks and manipulation of spatial data. Further research is related to the development of methods for automatic generalization of use cases and the integration of additional contextual analyzers.

• ALTERNATIVE APPROACHES TO NLP MODEL SCALE-UP: AN ANALYSIS OF APPROACHES TO OPTIMIZING DATA AND COMPUTATION VOLUME WHEN TRAINING LARGE-SCALE LANGUAGE MODELS

  К.I. Ralko , N. Е. Sergeev

  152-172

  2026-07-07

  Abstract ▼

  This paper focuses on overcoming the systemic limitations of the large-scale language model (LLM) scaling paradigm, which are related to data exhaustion and exponential growth in computational costs. This enables the development of more efficient approaches to building NLP models without sacrificing their performance. The goal of this study is to compare the performance of a standard transformer architecture (nanoGPT) and a model using semantic embeddings (nanoSonar) for language modeling tasks under resource constraints. Working with conceptual embeddings allows us to identify deeper linguistic patterns and reduce the amount of required training data, significantly improving modeling efficiency. The study utilized the TinyStories dataset, which includes short narratives with a clear structure. Before implementing the models, the data was preprocessed: for nanoGPT, tokenization was performed using the BPE method, and for nanoSonar, text was converted into semantic embeddings using a pretrained Sonar model. The models were evaluated using the loss and perplexity metrics. The results showed that the nanoSonar model provides significantly lower perplexity (6.609 versus 39.151 for nanoGPT) and demonstrates more robust training dynamics at later stages. This paper presents an analysis of modern approaches to scaling optimization (MoE, distillation, PEFT) and promising architectures (LRM, SSM, RWKV), and provides practical recommendations for applying models operating in the space of semantic embeddings to domain-specific problems and systems with limited computational resources. The results of this study can be useful in developing efficient language models that combine high generation quality with a cost-effective architecture.

• DEVELOPMENT OF A CHATBOT FOR CLASSIFICATION AND ANALYSIS OF NATURAL LANGUAGE TEXTS USING LOCAL LARGE LANGUAGE MODELS

  Juman Hussain Mohammad , Juman Hussain Mohammad , Y.А. Kravchenko

  159-171

  2025-07-24

  Abstract ▼

  This paper explores local large language models (LLMs) and their application in text classification tasks, while also comparing their performance with traditional methods. The paper provides a comprehensive review of several key local LLMs, with particular focus on their architectural advantages, characteristics, and application domains. Specifically, we examine models with varying numbers of parameters, their ability to adapt to specialized domains, and their computational requirements when deployed on local hardware. Special emphasis is placed on the trade-offs between performance and resource efficiency. As a practical contribution, we developed a chatbot that utilizes local LLMs (such as DeepSeek, Gemma, and Llama2 via Ollama) to classify incoming texts into predefined categories, demonstrating the operation of these models without cloud computing. The system features a modular architecture that allows for easy integration of new models and comparison of their effectiveness. The computational experiment involves evaluating the accuracy and inference speed of local LLMs compared to simpler methods such as Sentence-BERT, TF-IDF and BoWC, highlighting scenarios in which local models outperform or underperform traditional approaches. Testing was conducted using the benchmark BBC dataset. The results show that language models (including 7-billion parameter models) demonstrate strong and logically consistent classification performance in natural language text processing. However, their results are not perfect for benchmark datasets. Notably, we identified cases where all tested models, including traditional methods, misclassified documents, suggesting potential issues with data labeling. These findings indicate the need to reconsider benchmark labels in standard datasets, particularly for domains with subjective categories where expert evaluations may vary significantly. On the other hand, while local LLMs lag behind cloud-based solutions in speed, their advantages in data privacy and offline operation make them suitable for specialized tasks. This is particularly valuable in medical and financial institutions where protection of sensitive information is critical, and where local models can be fine-tuned for specific business processes without the constraints of cloud APIs.