The automatic extraction of causal relations is critically important for decision support systems, but its development is hindered by the scarcity of annotated corpora. The aim of this study is to comparatively analyze the effectiveness of three types of training data: open expert corpora, a manually annotated political corpus, and synthetic data generated by a large language model. Experiments were conducted using the DistilBERT architecture with token-level BIOES tagging. Synthetic data were generated with the Llama-3 LLM by encapsulating causes and effects in XML markers, followed by deterministic conversion into the BIOES format. For comparison, two expert corpora (EventStoryLine and SemEval-2010 Task 8), the expert-annotated political corpus PolitiCAUSE, and a synthetic sample of 1,100 balanced examples were used. The model trained on synthetic data achieved a Macro F1 score of 0.736, which is 6.7% higher than the result obtained from training on the expert corpora (0.690). Training on the political corpus PolitiCAUSE (24,417 examples) yielded a substantially lower result, with a Macro F1 score of 0.367. This discrepancy is attributed to class imbalance (approximately 15% causal tokens versus 50% in the synthetic sample) and to differences in task formulation: PolitiCAUSE is oriented toward binary sentence-level classification of causal presence, whereas the present study addresses token-level span labeling. The hypothesis that annotation quality and class balance are more important than data volume is confirmed. The proposed LLM-generation pipeline makes it possible to create training datasets that outperform expert corpora without direct manual annotation costs. The method is recommended for rapid deployment of causal relation extraction systems in new domains.
This article addresses the problem of developing a causal full-tuples extraction model from unstructured texts to represent decision-making situations in complex social and humanitarian environments. We present a causal full-tuples extraction model using a pre-trained BERT with additional feature-based special fine-tuning. To refine the causal classification, the model uses two types of features (verb causality and cause-and-effect quality metrics) to recognize a causal tuple, automatically extracts semantic features from sentences, increasing the accuracy of extraction. Text preprocessing is performed using the open source SpaCy library. The extracted cause-and-effect tuples in the format <cause phrase, verb phrase, effect phrase, polarity> are easily transformed into the corresponding elements of the graph <outgoing graph node, graph arc direction, incoming graph node, graph connection weight sign> and can then be used to construct a directed weighted signed graph with deterministic causality on arcs. In order to reduce dependence on external knowledge, synthetic generated annotated datasets are used to fine-tune and test the BERT model. Experimental results show that the accuracy of extracting cause-and-effect relationships on synthetic data reaches 94%, and the F1 value is 95%. The advantages of the presented technological solution are that the model does not require high operating costs, is implemented on a computer with standard characteristics, uses free software, which makes it accessible to a wide variety of users. It is expected that the proposed model can be used to automate text analysis and support decision-making in conditions of high uncertainty, which is especially important for social and humanitarian environments.
