Online Optimal Control of Robotic Systems with Single Critic NN-Based Reinforcement Learning
This paper suggests an online solution for the optimal tracking control of robotic systems based on a single critic neural network (NN)-based reinforcement learning (RL) method. To this end, we rewrite the robotic system model as a state-space form, which will facilitate the realization of optimal t...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2021-01-01
|
| Series: | Complexity |
| Online Access: | http://dx.doi.org/10.1155/2021/8839391 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849685775273164800 |
|---|---|
| author | Xiaoyi Long Zheng He Zhongyuan Wang |
| author_facet | Xiaoyi Long Zheng He Zhongyuan Wang |
| author_sort | Xiaoyi Long |
| collection | DOAJ |
| description | This paper suggests an online solution for the optimal tracking control of robotic systems based on a single critic neural network (NN)-based reinforcement learning (RL) method. To this end, we rewrite the robotic system model as a state-space form, which will facilitate the realization of optimal tracking control synthesis. To maintain the tracking response, a steady-state control is designed, and then an adaptive optimal tracking control is used to ensure that the tracking error can achieve convergence in an optimal sense. To solve the obtained optimal control via the framework of adaptive dynamic programming (ADP), the command trajectory to be tracked and the modified tracking Hamilton-Jacobi-Bellman (HJB) are all formulated. An online RL algorithm is the developed to address the HJB equation using a critic NN with online learning algorithm. Simulation results are given to verify the effectiveness of the proposed method. |
| format | Article |
| id | doaj-art-c6ffdbea93a74f06b51aed4f3d066232 |
| institution | DOAJ |
| issn | 1076-2787 1099-0526 |
| language | English |
| publishDate | 2021-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | Complexity |
| spelling | doaj-art-c6ffdbea93a74f06b51aed4f3d0662322025-08-20T03:22:58ZengWileyComplexity1076-27871099-05262021-01-01202110.1155/2021/88393918839391Online Optimal Control of Robotic Systems with Single Critic NN-Based Reinforcement LearningXiaoyi Long0Zheng He1Zhongyuan Wang2School of Computer Science, Wuhan University, Wuhan 430072, ChinaSchool of Computer Science, Wuhan University, Wuhan 430072, ChinaSchool of Computer Science, Wuhan University, Wuhan 430072, ChinaThis paper suggests an online solution for the optimal tracking control of robotic systems based on a single critic neural network (NN)-based reinforcement learning (RL) method. To this end, we rewrite the robotic system model as a state-space form, which will facilitate the realization of optimal tracking control synthesis. To maintain the tracking response, a steady-state control is designed, and then an adaptive optimal tracking control is used to ensure that the tracking error can achieve convergence in an optimal sense. To solve the obtained optimal control via the framework of adaptive dynamic programming (ADP), the command trajectory to be tracked and the modified tracking Hamilton-Jacobi-Bellman (HJB) are all formulated. An online RL algorithm is the developed to address the HJB equation using a critic NN with online learning algorithm. Simulation results are given to verify the effectiveness of the proposed method.http://dx.doi.org/10.1155/2021/8839391 |
| spellingShingle | Xiaoyi Long Zheng He Zhongyuan Wang Online Optimal Control of Robotic Systems with Single Critic NN-Based Reinforcement Learning Complexity |
| title | Online Optimal Control of Robotic Systems with Single Critic NN-Based Reinforcement Learning |
| title_full | Online Optimal Control of Robotic Systems with Single Critic NN-Based Reinforcement Learning |
| title_fullStr | Online Optimal Control of Robotic Systems with Single Critic NN-Based Reinforcement Learning |
| title_full_unstemmed | Online Optimal Control of Robotic Systems with Single Critic NN-Based Reinforcement Learning |
| title_short | Online Optimal Control of Robotic Systems with Single Critic NN-Based Reinforcement Learning |
| title_sort | online optimal control of robotic systems with single critic nn based reinforcement learning |
| url | http://dx.doi.org/10.1155/2021/8839391 |
| work_keys_str_mv | AT xiaoyilong onlineoptimalcontrolofroboticsystemswithsinglecriticnnbasedreinforcementlearning AT zhenghe onlineoptimalcontrolofroboticsystemswithsinglecriticnnbasedreinforcementlearning AT zhongyuanwang onlineoptimalcontrolofroboticsystemswithsinglecriticnnbasedreinforcementlearning |