基于强化学习的柴油机调速算法研究
Research on Diesel Engine Speed Regulation Algorithm Based on Reinforcement Learning
关键词:柴油机调速  PID控制器  强化学习算法  非线性复杂系统
Key Words:diesel engine speed regulation  PID controller  reinforcement learning algorithm  nonlinear complex systems
基金项目:检测技术与节能装置安徽省重点实验室开放基金 No. JCKJ2022A01
作者单位邮编
姚崇 哈尔滨工程大学 动力与能源工程学院 150001
董璕 哈尔滨工程大学 动力与能源工程学院 
李瑞* 哈尔滨工程大学 动力与能源工程学院 150001
龙云 哈尔滨工程大学 动力与能源工程学院 
宋恩哲 哈尔滨工程大学 动力与能源工程学院 
摘要:柴油机转速控制问题属于非线性复杂系统问题。为了更好地调节柴油机转速，笔者首次提出一种强化学习-PID控制器,并应用到了柴油机转速控制中。首先,基于连续动作空间的SAC(Soft Actor-Critic)算法,结合连续型PID控制器,本文设计了一种强化学习-PID控制器,可代替传统PID控制的转速环。优化设计了基于actor-critic(行动者评价者)框架的输入输出和奖励函数以匹配柴油机特性,采用随机动作增加寻优效率,形成SAC-PID控制柴油机转速的网络交互结构,达到快速调整转速,减小稳定时间的效果；其次,构建了以柴油机D6114为原型机的MATLAB/Simulink平均值模型,并利用实验数据验证了仿真模型的有效性；最后,利用平均值模型,仿真了本文提出的控制算法效果。经过仿真验证本文所提出的算法使柴油机转速响应曲线超调量更小、响应时间更快、鲁棒性更强,加载时瞬态调速率减小1.28%,减载时瞬态调速率减小0.92%,加载时稳定时间减少0.4s,减载时稳定时间减少3.6s,验证SAC-PID控制负载瞬态调速率和稳定时间均已达到1级精度指标；最后,仿真对比验证了SAC算法的联合控制效果较其他算法最佳。
Abstract:The speed control problem of diesel engines belongs to nonlinear and complex system problems.In order to better regulate the speed of diesel engines, the author proposes for the first time a reinforcement learning PID controller and applies it to diesel engine speed control. Firstly, based on the SAC (Soft Actor-Critic) algorithm with continuous action space, combined with a continuous PID controller, this paper designs a reinforcement learning PID controller that can replace the speed loop of traditional PID control. Optimized the design of input-output and reward functions based on the actor critic framework to match the characteristics of diesel engines. Random actions were used to increase optimization efficiency, forming a network interaction structure for SAC-PID control of diesel engine speed, achieving the effect of quickly adjusting speed and reducing stabilization time; Secondly, a MATLAB/Simulink average model was constructed using the diesel engine D6114 as the prototype, and the effectiveness of the simulation model was verified using experimental data; Finally, the effectiveness of the control algorithm proposed in this paper was simulated using the average value model. After simulation verification, the algorithm proposed in this article reduces the overshoot of the diesel engine speed response curve, responds faster, and has stronger robustness. The transient speed regulation rate during loading is reduced by 1.28%, the transient speed regulation rate during unloading is reduced by 0.92%, the stability time during loading is reduced by 0.4s, and the stability time during unloading is reduced by 3.6s. It is verified that the SAC-PID control load transient speed regulation rate and stability time have reached the first level accuracy index; Finally, simulation comparison verified that the SAC algorithm has the best joint control effect compared to other algorithms.
查看/发表评论