DEVELOPMENT OF A CHATBOT FOR CLASSIFICATION AND ANALYSIS OF NATURAL LANGUAGE TEXTS USING LOCAL LARGE LANGUAGE MODELS
Abstract
This paper explores local large language models (LLMs) and their application in text classification tasks, while also comparing their performance with traditional methods. The paper provides a comprehensive review of several key local LLMs, with particular focus on their architectural advantages, characteristics, and application domains. Specifically, we examine models with varying numbers of parameters, their ability to adapt to specialized domains, and their computational requirements when deployed on local hardware. Special emphasis is placed on the trade-offs between performance and resource efficiency. As a practical contribution, we developed a chatbot that utilizes local LLMs (such as DeepSeek, Gemma, and Llama2 via Ollama) to classify incoming texts into predefined categories, demonstrating the operation of these models without cloud computing. The system features a modular architecture that allows for easy integration of new models and comparison of their effectiveness. The computational experiment involves evaluating the accuracy and inference speed of local LLMs compared to simpler methods such as Sentence-BERT, TF-IDF and BoWC, highlighting scenarios in which local models outperform or underperform traditional approaches. Testing was conducted using the benchmark BBC dataset. The results show that language models (including 7-billion parameter models) demonstrate strong and logically consistent classification performance in natural language text processing. However, their results are not perfect for benchmark datasets. Notably, we identified cases where all tested models, including traditional methods, misclassified documents, suggesting potential issues with data labeling. These findings indicate the need to reconsider benchmark labels in standard datasets, particularly for domains with subjective categories where expert evaluations may vary significantly. On the other hand, while local LLMs lag behind cloud-based solutions in speed, their advantages in data privacy and offline operation make them suitable for specialized tasks. This is particularly valuable in medical and financial institutions where protection of sensitive information is critical, and where local models can be fine-tuned for specific business processes without the constraints of cloud APIs.
References
1. OpenAI, J. Achiam, S. Adler, S. Agarwal, L. Ahmad, B. Zoph [et al.]. OpenAI. GPT-4 Technical Re-port, 2024.
2. Baktash J.A., Dawodi M. Gpt-4: A Review on Advancements and Opportunities in Natural Language Processing, 2023.
3. Allahyari M., Pouriyeh S., Assefi M., Safaei S., Trippe E.D., Gutierrez J.B., Kochut K. A brief survey of text mining: Classification, clustering and extraction techniques, 2017.
4. Roumeliotis K.I., Tselikas N.D., Nasiopoulos D.K. Llama 2: Early Adopters’ Utilization of Meta’s New Open-Source Pretrained Model, 2023.
5. DeepSeek-AI., Guo D., Yang D., Zhang H., Song J., Zhang Z. [et al.]. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning, 2025.
6. Zhang C., Deng Y., Lin X., Wang B., Ng D., Ye H., Li X., Xiao Y., Mo Z., Zhang Q., Bing L. 100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Models, 2025.
7. Team G., Mesnard T., Hardin C., Dadashi R., Bhupatiraju S., Kenealy K. [et al.]. Gemma: Open Mod-els Based on Gemini Research and Technology, 2024.
8. Reimers N., Gurevych I. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks, arXiv preprint arXiv, 2019, Vol. abs/1908.10084.
9. Mansour A., Mohammad J., Kravchenko Y., Kravchenko D., Silega N. Harnessing Key Phrases in Con-structing a Concept-Based Semantic Representation of Text Using Clustering Techniques, International Workshop on Artificial Intelligence and Pattern Recognition. Springer, 2023, pp. 190-201.
10. Mansour A., Mohammad J., Kravchenko Y. Text Vectorization Method Based on Concept Mining Us-ing Clustering Techniques, 2022 VI International Conference on Information Technologies in Engineer-ing Education (Inforino). IEEE, 2022, pp. 1-10.
11. Mansour A.M., Mohammad J.H., Kravchenko Y.A. Text vectorization using data mining methods, Izves-tia SFedU. Technical science, 2021, No. 2.
12. Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A.N., Kaiser Ł., Polosukhin I. At-tention is all you need, Advances in neural information processing systems, 2017, Vol. 30.
13. Franceschelli G., Musolesi M. Creative Beam Search: LLM-as-a-Judge For Improving Response Gen-eration, 2024.
14. Pryzant R., Iter D., Li J., Lee Y.T., Zhu C., Zeng M. Automatic Prompt Optimization with “Gradient Descent” and Beam Search, 2023.
15. Adeshina A.A. Building Python Web APIs with FastAPI: A fast-paced guide to building high-performance, robust web APIs with very little boilerplate code. Packt Publishing Ltd, 2022.
16. Giray L. Prompt engineering with ChatGPT: a guide for academic writers, Annals of biomedical engi-neering, 2023, Vol. 51, No. 12. pp. 2629-2633.
17. Marvin G., Hellen N., Jjingo D., Nakatumba-Nabende J. Prompt Engineering in Large Language Mod-els, Data Intelligence and Cognitive Informatics: Algorithms for Intelligent Systems, eds. I.J. Jacob, S. Piramuthu, P. Falkowski-Gilski. Singapore: Springer Nature Singapore, 2024, pp. 387-402. ISBN 978-981-9979-99-8.
18. Mahmoud Bsharat S., Myrzakhan A., Shen Z. Principled Instructions Are All You Need for Question-ing LLaMA-1/2, GPT-3.5/4, arXiv e-prints, 2023, pp. arXiv-2312.
19. Mann B., Ryder N., Subbiah M., Kaplan J., Dhariwal P., Neelakantan A., Shyam P., Sastry G., Askell A., Agarwal S. Language models are few-shot learners, arXiv preprint arXiv:2005.14165, 2020,
Vol. 1, pp. 3.
20. Sabbah T., Selamat A., Selamat M.H., Al-Anzi F.S., Viedma E.H., Krejcar O., Fujita H. Modified fre-quency-based term weighting schemes for text classification, Applied Soft Computing, 2017, Vol. 58, pp. 193-206.
