ASSESSMENT OF THE INFLUENCE OF NEURAL NETWORK HYPERPARAMETERS ON THE ACCURACY OF FORECASTING ENERGY CONSUMPTION
Abstract
The work is devoted to the problem of improving the accuracy of short-term forecasting of electricity
consumption using deep machine learning tools. The influence of the specified neural network NN
hyperparameters on the error in predicting power consumption, such as: data packet size – Bs; number of
NN layers – j; neuron activation functions – Fa; optimizers – O. The optimal hyperparameters of the NN
model for predicting electrical consumption (EC) for consumers of additive and cyclic types have been
determined. The analysis of the impact of the batch size on the accuracy of the forecast showed an increase
in the effectiveness of NN training with the growth of the input data package. The analysis of the
influence of the number of layers showed that with an increase in the number of layers of the NN, the
learning time decreases and its predictions become more accurate. A study of various optimizers for
learning speed has shown that the best results are demonstrated by the optimizers “Adam” and
“RMSProp". It is shown that the choice of the activation function determines how quickly the NN will be
trained and how accurate its forecasts will be. The use of various regularization methods allows NS to
achieve better results in practice, improving their generalization ability and increasing the accuracy of
predictions. It is shown that in order to achieve the minimum forecasting error, it is necessary to individually
configure the network parameters for each consumer, taking into account significant differences in
the nature of energy consumption. The training and testing of the created network with selected parameters
was carried out on a training and test sample containing data on electricity consumption for 2 years
(17520 hours). The analysis of input data on power consumption showed that the optimal parameters of the predictive neural network model in manual mode are: package size 250 (selected empirically), 5 layers,
activation function “ReLU", optimizer “Adam". Various ways of selecting hyperparameters (manually
and by means of genetic algorithm (GA)) are considered.
References
prognozirovaniya resursa izolyatsionnykh materialov silovykh kabel'nykh liniy [Algorithm for processing
an artificial neural network for factor forecasting of the resource of insulating materials of
power cable lines], Izvestiya YuFU. Tekhnicheskie nauki [Izvestiya SFedU. Engineering Sciences],
2021, No. 2, pp. 59-73.
2. Poluyanovich N.K., Dubyago M.N. Otsenka vozdeystvuyushchikh faktorov i prognozirovanie elektropotrebleniya
v regional'noy energosisteme s uchetom rezhima ee ekspluatatsii [Assessment of the current
factors and forecasting of power consumption in the regional power system, taking into account
its operating mode], Izvestiya YuFU. Tekhnicheskie nauki [Izvestiya SFedU. Engineering Sciences],
2022, No. 2 (226), pp. 31-46.
3. Poluyanovich N.K., Dubyago M.N., Bur'kov D.V. Termofluktuatsionnaya teoriya razrusheniya i
otsenka dolgovechnosti elektricheskoy izolyatsii SPE kabeley [Thermal fluctuation theory of destruction
and durability assessment-news of electrical insulation of SPE cables], Izvestiya YuFU.
Tekhnicheskie nauki [Izvestiya SFedU. Engineering Sciences], 2021, No. 6, pp. 180-194.
4. Poluyanovich N.K., Dubyago M.N. Book Chapter. Structural Changes During Electrical Aging of Insulation
Materials of Cable Networks, Studies in Systems, Decision and Control, 2023, 477, pp 245-256.
Available at: https://doi.org/10.1007/978-3-031-33159-6_19.
5. Poluyanovich N.K., Sharykin A.A., Dubyago M.N., Kachelaev O.V. Conference Paper. Analysis of the
Electro-magnetic Field on Cable Systems with Insulation Made of Polymer Materials, Proceedings -
2023 5th International Conference on Control Systems, Mathematical Modeling, Automation and Energy
Efficiency, SUMMA 2023, 2023, pp. 977-982.
6. Shiyan Ren; Siwei Guo; Xiaojun Liu; Qingxin Liu. Shielding Effectiveness of Double-layer Magnetic
Shield of Current Comparator Under Radial Disturbing Magnetic Field, IEEE Transactions on Magnetics,
2016, Vol. PP, Issue 99, pp. 1.
7. IEE Guide for Field Testing of Shielded Power Cable Systems Using Very Low Frequency (VLF)
(less than 1 Hz). IEE.400.2, 2013.
8. Ukil A., Braendle H., Krippner P. Distributed temperature sensing: review of technology and applications,
Sensors Journal, IEEE, 2012, Vol. 12, No. 5, pp. 885-892.
9. Diaz-Aguilo M., De Leon F. Introducing Mutual Heating Effects in the Lad-der-Type Soil Model for
Dynamic Thermal Rating of Underground Cables, IEEE Tran. on Power Delivery, 2015, Vol. 30 (4),
pp. 1958-1964.
10. Poluyanovich N.K., Dubyago M.N., Azarov N.V., Ogrenichev A.V., Sharykin A.А. Multifactor Model for
Forecasting Thermal Processes in the Insulating Materials of Cable Lines // Сonference proceedings // 2020
International Multi-Conference on Indus-trial Engineering and Modern Technologies, FarEastCon. – 2020.
– P. 9271367. – ISBN: 978-172816951-4. – DOI: 10.1109/FarEastCon50210.2020.9271367.
11. Postanovlenie Pravitel'stva RF ot 9 sentyabrya 2023 g. N 1473 "Ob utverzhdenii kompleksnoy
gosudarstvennoy programmy Rossiyskoy Federatsii "Energosberezhenie i povyshenie energeticheskoy
effektivnosti" [Decree of the Government of the Russian Federation No. 1473 dated September 9,
2023 "On approval of the comprehensive State program of the Russian Federation "Energy Saving and
energy Efficiency improvement"], Razmeshchen v SPS "Konsul'tant Plyus" [Posted in the SPS Consultant
Plus]. Available at: https://base.garant.ru/407632842/.
12. Dubyago M.N., Poluyanovich N.K. Sovershenstvovanie metodov diagnostiki i prognozirovaniya
elektroizolyatsionnykh materialov sistem energosnabzheniya: monografiya [Improvement of methods
of diagnostics and forecasting of electrical insulating materials of power supply systems: monograph].
Rostov-on-Don; Taganrog: Izd-vo YuFU, 2019, 192 p.
13. Sergeev N.N., Matrenin P.V. Enhancing Efficiency of Ensemble Machine Learning Models for Short-
Term Load Forecasting through Feature Selection, 2022 IEEE 23rd International Conference of Young
Professionals in Electron Devices and Materials (EDM): Proc., Altai, 30 June – 04 July 2022. Altai:
IEEE, 2022, pp. 368-371.
14. Rafi S.H., Nahid-Al-Masood, Deeba S.R., Hossain E. A short-term power load forecasting method
using integrated CNN and LSTM network, IEEE Access, 2021, Vol. 9, pp. 32436-32448.
15. Deng Z., Wang B., Xu Y. [et al.]. Multi-scale convolutional neural network with time-cognition for
multi-step short- term power load forecasting, IEEE Access, 2019, Vol. 7, pp. 88058-88071.
16. Vyalkova S.A., Nadtoka I.I., Kornyukova O.A. Primenenie neyronnykh setey dlya prognozirovaniya
elektropotrebleniya megapolisa [Application of neural networks for forecasting the power consumption
of a megalopolis], Russian Internet Journal of Industrial Engineering, 2023, Vol. 10, No. 4.
17. Polyakhov N.D., Prikhod'ko I.A., Van Efen. Prognozirovanie elektropotrebleniya na osnove metoda
opornykh vektorov s ispol'zovaniem evolyutsionnykh algoritmov optimizatsii [Forecasting of power consumption
based on the method of support vectors using evolutionary optimization algorithms],
Sovremennye problemy nauki i obrazovaniya [Modern problems of science and education], 2013, No. 2.
18. Qiu X., Zhang L., Ren Y., Suganthan P., Amaratunga G. Ensemble deep learning for regression and
time series forecasting, Symposium on Computational Intelligence in Ensemble Learning. IEEE, 2014,
pp. 21-26. DOI: 10.1109/CIEL.2014.7015739.
19. Kong W., Dong Z., Jia Y., Hill D., Xu Y., Zhang Y. Short-term residential load forecasting based on
LSTM recurrent neural network, IEEE Transactions on Smart Grid, 2019, Vol. 10, No. 1, pp. 841-851.
DOI: 10.1109/TSG.2017.2753802.
20. Ryu S., Noh J., Kim H. Deep neural network based demand side short term load forecasting, Energies,
2016, Vol. 10, No. 1, pp. 1-20. DOI: 10.3390/en10010003.
21. Chen K., Wang Q., He Z., Hu J., He J. Short-Term Load Forecasting with Deep Residual Networks,
IEEE Trans. Smart Grid, 2019, Vol. 10, No. 4, pp. 3943-3952. DOI: 10.1109/TSG.2018.2844307.
22. Khomutov S.O., Serebryakov N.A. Sozdanie neyrosetevoy matematicheskoy modeli kratkosrochnogo
prognozirovaniya elektropotrebleniya elektrotekhnicheskogo kompleksa uchastka rayonnykh
elektricheskikh setey 6–35 kV [Creation of a neural network mathematical model for short-term forecasting
of electrical consumption of an electrotechnical complex of a section of district electric networks
6-35 kV], Transportnye sistemy i tekhnologii [Transport systems and technologies], 2020,
Vol. 6, No. 1, pp. 80-91. DOI: 10.17816/ transsyst20206180-91.
23. Osnovnye funktsii aktivatsii v neyronnykh setyakh [Basic activation functions in neural networks],
Stat'i po programmirovaniyu [Articles on programming]. Available at:
https://www.yourtodo.ru/posts/osnovnyie-funktsii-aktivatsii-v-nejronnyih-setyah/ (04 July 2024).
24. Priemy povysheniya skhodimosti neyronnykh setey. Regulyarizatsiya [Techniques for increasing the
convergence of neural networks. Regularization], MQL5; treyding, avtomaticheskie torgovye sistemy,
testirovanie strategiy i tekhnicheskie indikatory na MetaTrader [MQL5; trading, automatic trading
systems, strategy testing and technical indicators on MetaTrader]. Available at:
https://www.mql5.com/ru/neurobook/index/about_ai/improvement/regularization/ (04 July 2024).
