LSTM-Enhanced Deep Reinforcement Learning for Robust Trajectory Tracking Control of Skid-Steer Mobile Robots Under Terra-Mechanical Constraints
Autonomous navigation in mining environments is challenged by complex wheel–terrain interaction, traction losses caused by slip dynamics, and sensor limitations. This paper investigates the effectiveness of Deep Reinforcement Learning (DRL) techniques for the trajectory tracking control of skid-stee...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-05-01
|
| Series: | Robotics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2218-6581/14/6/74 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849704786181488640 |
|---|---|
| author | Jose Manuel Alcayaga Oswaldo Anibal Menéndez Miguel Attilio Torres-Torriti Juan Pablo Vásconez Tito Arévalo-Ramirez Alvaro Javier Prado Romo |
| author_facet | Jose Manuel Alcayaga Oswaldo Anibal Menéndez Miguel Attilio Torres-Torriti Juan Pablo Vásconez Tito Arévalo-Ramirez Alvaro Javier Prado Romo |
| author_sort | Jose Manuel Alcayaga |
| collection | DOAJ |
| description | Autonomous navigation in mining environments is challenged by complex wheel–terrain interaction, traction losses caused by slip dynamics, and sensor limitations. This paper investigates the effectiveness of Deep Reinforcement Learning (DRL) techniques for the trajectory tracking control of skid-steer mobile robots operating under terra-mechanical constraints. Four state-of-the-art DRL algorithms, i.e., Proximal Policy Optimization (PPO), Deep Deterministic Policy Gradient (DDPG), Twin Delayed DDPG (TD3), and Soft Actor–Critic (SAC), are selected to evaluate their ability to generate stable and adaptive control policies under varying environmental conditions. To address the inherent partial observability in real-world navigation, this study presents an original approach that integrates Long Short-Term Memory (LSTM) networks into DRL-based controllers. This allows control agents to retain and leverage temporal dependencies to infer unobservable system states. The developed agents were trained and tested in simulations and then assessed in field experiments under uneven terrain and dynamic model parameter changes that lead to traction losses in mining environments, targeting various trajectory tracking tasks, including lemniscate and squared-type reference trajectories. This contribution strengthens the robustness and adaptability of DRL agents by enabling better generalization of learned policies compared with their baseline counterparts, while also significantly improving trajectory tracking performance. In particular, LSTM-based controllers achieved reductions in tracking errors of 10%, 74%, 21%, and 37% for DDPG-LSTM, PPO-LSTM, TD3-LSTM, and SAC-LSTM, respectively, compared with their non-recurrent counterparts. Furthermore, DDPG-LSTM and TD3-LSTM reduced their control effort through the total variation in control input by 15% and 20% compared with their respective baseline controllers, respectively. Findings from this work provide valuable insights into the role of memory-augmented reinforcement learning for robust motion control in unstructured and high-uncertainty environments. |
| format | Article |
| id | doaj-art-e9f95e4c104e4e72b99e73c8055fe525 |
| institution | DOAJ |
| issn | 2218-6581 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Robotics |
| spelling | doaj-art-e9f95e4c104e4e72b99e73c8055fe5252025-08-20T03:16:39ZengMDPI AGRobotics2218-65812025-05-011467410.3390/robotics14060074LSTM-Enhanced Deep Reinforcement Learning for Robust Trajectory Tracking Control of Skid-Steer Mobile Robots Under Terra-Mechanical ConstraintsJose Manuel Alcayaga0Oswaldo Anibal Menéndez1Miguel Attilio Torres-Torriti2Juan Pablo Vásconez3Tito Arévalo-Ramirez4Alvaro Javier Prado Romo5Departamento de Ingeniería de Sistemas y Computación, Universidad Católica del Norte, Antofagasta 1249004, ChileDepartamento de Ingeniería de Sistemas y Computación, Universidad Católica del Norte, Antofagasta 1249004, ChileDepartment of Electrical Engineering, School of Engineering, Faculty of Engineering, Pontificia Universidad Católica de Chile, Santiago 7820436, ChileEnergy Transformation Center, Faculty of Engineering, Universidad Andrés Bello, Santiago 7500000, ChileDepartment of Mechanical Engineering and Metallurgy, School of Engineering, Faculty of Engineering, Pontificia Universidad Católica de Chile, Santiago 7820436, ChileDepartamento de Ingeniería de Sistemas y Computación, Universidad Católica del Norte, Antofagasta 1249004, ChileAutonomous navigation in mining environments is challenged by complex wheel–terrain interaction, traction losses caused by slip dynamics, and sensor limitations. This paper investigates the effectiveness of Deep Reinforcement Learning (DRL) techniques for the trajectory tracking control of skid-steer mobile robots operating under terra-mechanical constraints. Four state-of-the-art DRL algorithms, i.e., Proximal Policy Optimization (PPO), Deep Deterministic Policy Gradient (DDPG), Twin Delayed DDPG (TD3), and Soft Actor–Critic (SAC), are selected to evaluate their ability to generate stable and adaptive control policies under varying environmental conditions. To address the inherent partial observability in real-world navigation, this study presents an original approach that integrates Long Short-Term Memory (LSTM) networks into DRL-based controllers. This allows control agents to retain and leverage temporal dependencies to infer unobservable system states. The developed agents were trained and tested in simulations and then assessed in field experiments under uneven terrain and dynamic model parameter changes that lead to traction losses in mining environments, targeting various trajectory tracking tasks, including lemniscate and squared-type reference trajectories. This contribution strengthens the robustness and adaptability of DRL agents by enabling better generalization of learned policies compared with their baseline counterparts, while also significantly improving trajectory tracking performance. In particular, LSTM-based controllers achieved reductions in tracking errors of 10%, 74%, 21%, and 37% for DDPG-LSTM, PPO-LSTM, TD3-LSTM, and SAC-LSTM, respectively, compared with their non-recurrent counterparts. Furthermore, DDPG-LSTM and TD3-LSTM reduced their control effort through the total variation in control input by 15% and 20% compared with their respective baseline controllers, respectively. Findings from this work provide valuable insights into the role of memory-augmented reinforcement learning for robust motion control in unstructured and high-uncertainty environments.https://www.mdpi.com/2218-6581/14/6/74deep reinforcement learningtrajectory trackingrecurrent neural networkterra-mechanical constraints |
| spellingShingle | Jose Manuel Alcayaga Oswaldo Anibal Menéndez Miguel Attilio Torres-Torriti Juan Pablo Vásconez Tito Arévalo-Ramirez Alvaro Javier Prado Romo LSTM-Enhanced Deep Reinforcement Learning for Robust Trajectory Tracking Control of Skid-Steer Mobile Robots Under Terra-Mechanical Constraints Robotics deep reinforcement learning trajectory tracking recurrent neural network terra-mechanical constraints |
| title | LSTM-Enhanced Deep Reinforcement Learning for Robust Trajectory Tracking Control of Skid-Steer Mobile Robots Under Terra-Mechanical Constraints |
| title_full | LSTM-Enhanced Deep Reinforcement Learning for Robust Trajectory Tracking Control of Skid-Steer Mobile Robots Under Terra-Mechanical Constraints |
| title_fullStr | LSTM-Enhanced Deep Reinforcement Learning for Robust Trajectory Tracking Control of Skid-Steer Mobile Robots Under Terra-Mechanical Constraints |
| title_full_unstemmed | LSTM-Enhanced Deep Reinforcement Learning for Robust Trajectory Tracking Control of Skid-Steer Mobile Robots Under Terra-Mechanical Constraints |
| title_short | LSTM-Enhanced Deep Reinforcement Learning for Robust Trajectory Tracking Control of Skid-Steer Mobile Robots Under Terra-Mechanical Constraints |
| title_sort | lstm enhanced deep reinforcement learning for robust trajectory tracking control of skid steer mobile robots under terra mechanical constraints |
| topic | deep reinforcement learning trajectory tracking recurrent neural network terra-mechanical constraints |
| url | https://www.mdpi.com/2218-6581/14/6/74 |
| work_keys_str_mv | AT josemanuelalcayaga lstmenhanceddeepreinforcementlearningforrobusttrajectorytrackingcontrolofskidsteermobilerobotsunderterramechanicalconstraints AT oswaldoanibalmenendez lstmenhanceddeepreinforcementlearningforrobusttrajectorytrackingcontrolofskidsteermobilerobotsunderterramechanicalconstraints AT miguelattiliotorrestorriti lstmenhanceddeepreinforcementlearningforrobusttrajectorytrackingcontrolofskidsteermobilerobotsunderterramechanicalconstraints AT juanpablovasconez lstmenhanceddeepreinforcementlearningforrobusttrajectorytrackingcontrolofskidsteermobilerobotsunderterramechanicalconstraints AT titoarevaloramirez lstmenhanceddeepreinforcementlearningforrobusttrajectorytrackingcontrolofskidsteermobilerobotsunderterramechanicalconstraints AT alvarojavierpradoromo lstmenhanceddeepreinforcementlearningforrobusttrajectorytrackingcontrolofskidsteermobilerobotsunderterramechanicalconstraints |