The relevance of the problem of adaptive control of nonlinear dynamic objects is ensured by the ever-increasing demands on the quality and operating conditions of technical systems. Automation and robotics lead to an increase in the complexity of the problems being solved and the need to adapt to structural and parametric uncertainty and external disturbances. In recent years, the use of reinforcement learning methods for the synthesis of adaptive control systems has gained popularity. These methods demonstrate effectiveness in controlling uncertain dynamic objects. However, there are two fundamental problems with the application of machine learning methods to control systems for dynamic objects. First, the need to ensure asymptotic stability of the desired trajectory of a closed-loop system limits the application of deep learning methods. Second, during the learning process, it is necessary that the intelligent controller does not generate controls that lead to state variables exceeding specified limits. The purpose of this article is to review and analyze recent advances in the application of reinforcement learning methods to the synthesis of adaptive control systems for nonlinear dynamic objects. Particular attention is given to Actor-Critic methods, which are structurally similar to adaptive control systems with self-adjusting parameters. Based on the analysis, the structure of an adaptive system with two-component control, including nominal and adaptive controllers, is proposed. An adaptive control algorithm is proposed, distinguished by the use of a modified Actor-Critic algorithm, distinguished by a new form of the delta error and the absence of a singularity in the neighborhood of zero. The proposed algorithm allows for a reduction in the number of adjustable parameters. The article presents conditions for Lyapunov stability of the zero-equilibrium position of a closed-loop system and an example of the synthesis and modeling of the proposed adaptive control algorithm
