Trajectory Tracking Control for Wheeled Mobile Robot System with Uncertain Nonlinear Model based on Integral Reinforcement Learning Algorithm

Trajectory Tracking Control for Wheeled Mobile Robot System with Uncertain Nonlinear Model based on Integral Reinforcement Learning Algorithm
  IJETT-book-cover           
  
© 2024 by IJETT Journal
Volume-72 Issue-5
Year of Publication : 2024
Author : Doan Van Hoa, Tran Duc Chuyen, Lai Khac Lai, Le Thi Thu Ha
DOI : 10.14445/22315381/IJETT-V72I5P130

How to Cite?

Doan Van Hoa, Tran Duc Chuyen, Lai Khac Lai, Le Thi Thu Ha, "Trajectory Tracking Control for Wheeled Mobile Robot System with Uncertain Nonlinear Model based on Integral Reinforcement Learning Algorithm," International Journal of Engineering Trends and Technology, vol. 72, no. 5, pp. 290-298, 2024. Crossref, https://doi.org/10.14445/22315381/IJETT-V72I5P130

Abstract
TA mobile robot is a type of robot that is capable of moving on its own and performing tasks without human intervention. Mobile robots are equipped with sensors and control systems to detect and react to the surrounding environment. Designing a controller for mobile robots so that the working process achieves optimal performance is of interest to many scientists. In this study, the author proposes an Integral Reinforcement Learning (IRL) method combined with a disturbance observer to design a robust adaptive optimal controller to track the trajectory of the WMR system. The optimal controller uses a traditional Actor-Critic structure consisting of two neural networks, Critic NN and Actor NN. External disturbances and wheel slippage of the WMR are estimated by the Disturbance Observer (DO) and compensated for by the disturbance compensation controller. System simulation results on Matlab software show us the effectiveness of the proposed combined method.

Keywords
Reinforcement learning, Integral reinforcement learning, Actor-Critic, Wheeled mobile robot, Disturbance observer, Hamilton-Jacobi-Bellman.

References
[1] Jie Meng et al., “Two-Wheeled Robot Platform Based on PID Control,” 5th International Conference on Information Science and Control Engineering, Zhengzhou, China, pp. 1011-1014, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[2] R. Fierro, and F.L. Lewis, “Control of a Nonholomic Mobile Robot: Backstepping Kinematics into Dynamics,” Journal of Robotic Systems, vol. 14, no. 3, pp. 149-163, 1997.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Jun Ye, “Tracking Control for Nonholonomic Mobile Robots: Integrating the Analog Neural Network into the Backstepping Technique,” Neurocomputing, vol. 71, no. 16-18, pp. 3373-3378, 2008.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Shubhobrata Rudra, Ranjit Kumar Barai, and Madhubanti Maitra, “Design and Implementation of a Block-Backstepping Based Tracking Control for Nonholonomic Wheeled Mobile Robot,” International Journal of Robust and Nonlinear Control, vol. 26, no. 14, pp. 30183035, 2015.
[CrossRef] [Google Scholar] [Publisher Link]
[5] Xing Wu et al., “Backstepping Trajectory Tracking Based on Fuzzy Sliding Mode Control for Differential Mobile Robots,” Journal of Intelligent & Robotic Systems, vol. 96, pp. 109-121, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Bong Seok Park et al., “Adaptive Tracking Control of Nonholonomic Mobile Robots Considering Actuator Dynamics: Dynamic Surface Design Approach,” American Control Conference, St. Louis, MO, USA, pp. 3860-3865, 2009.
[CrossRef] [Google Scholar] [Publisher Link]
[7] Omid Mohareri, Rached Dhaouadi, and Ahmad B. Rad, “Indirect Adaptive Tracking Control of a Nonholonomic Mobile Robot via Neural Networks,” Neurocomputing, vol. 88, pp. 54-66, 2012.
[CrossRef] [Google Scholar] [Publisher Link]
[8] Altan Onat, and Metin Ozkan, “Dynamic Adaptive Trajectory Tracking Control of Nonholonomic Mobile Robots Using Multiple Models Approach,” Advanced Robotics, vol. 29, no. 14, pp. 913-928, 2015.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Mohammad Mehdi Fateh, and Aliasghar Arab, “Robust Control of a Wheeled Mobile Robot by Voltage Control Strategy,” Nonlinear Dynamics, vol. 79, no. 1, pp. 335-348, 2014.
[CrossRef] [Google Scholar] [Publisher Link]
[10] P. Navin Chandra, and S.J. Mija, “Robust Controller for Trajectory Tracking of a Mobile Robot,” IEEE 1st International Conference on Power Electronics, Intelligent Control and Energy Systems, Delhi, India, pp. 1-6, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[11] Spandan Roy et al., “Robust Control of Nonholonomic Wheeled Mobile Robot with Past Information: Theory and Experiment,” Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering, vol. 231, no. 3, pp. 178188, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[12] Chung-Hsun Sun, Yin-Tien Wang, and Cheng-Chung Chang, “Design of T-S Fuzzy Controller for Two-Wheeled Mobile Robot,” Proceedings of International Conference on System Science and Engineering, Macau, China, pp. 223-228, 2011.
[CrossRef] [Google Scholar] [Publisher Link]
[13] Min-Chi Kao et al., “Adaptive Type-2 Fuzzy Tracking Control of Wheeled Mobile Robots,” International Conference on Fuzzy Theory and Its Applications, Taipei, Taiwan, pp. 1-6, 2013.
[CrossRef] [Google Scholar] [Publisher Link]
[14] Qing Xu et al., “Fuzzy PID Based Trajectory Tracking Control of Mobile Robot and its Simulation in Simulink,” International Journal of Control and Automation, vol. 7, no. 8, pp. 233-244, 2014.
[CrossRef] [Google Scholar] [Publisher Link]
[15] Nacer Hacene, and Boubekeur Mendil, “Fuzzy Behavior-Based Control of Three Wheeled Omnidirectional Mobile Robot,” International Journal of Automation and Computing, vol. 16, pp. 163-185, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[16] Rafael Morales et al., “Robotics and Control Engineering of Wave and Tidal Energy-Recovering Systems,” Mathematical Problems in Engineering, vol. 2018, pp. 1-2, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[17] Mohamed Abdelwahab et al., “Trajectory Tracking of Wheeled Mobile Robots Using Z-Number Based Fuzzy Logic,” IEEE Access, vol. 8, pp. 18426-18441, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[18] Alexandr Štefek et al., “Optimization of Fuzzy Logic Controller Used for a Differential Drive Wheeled Mobile Robot,” Applied Sciences, vol. 11, no. 13, pp. 1-23, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[19] R. Fierro, and F.L. Lewis, “Control of a Nonholonomic Mobile Robot Using Neural Networks,” IEEE Transactions on Neural Networks, vol. 9, no. 4, pp. 589-600, 1998.
[CrossRef] [Google Scholar] [Publisher Link]
[20] Zhijun Li et al., “Trajectory-Tracking Control of Mobile Robot Systems Incorporating Neural-Dynamic Optimized Model Predictive Approach,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 46, no. 6, pp. 740-749, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[21] Pavol Bozek et al., “Neural Network Control of a Wheeled Mobile Robot Based on Optimal Trajectories,” International Journal of Advanced Robotic Systems, vol. 17, no. 2, pp. 1-10, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[22] Ziyu Chen et al., “Adaptive-Neural-Network-Based Trajectory Tracking Control for a Nonholonomic Wheeled Mobile Robot with Velocity Constraints,” IEEE Transactions on Industrial Electronics, vol. 68, no. 6, pp. 5057-5067, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[23] D. Vrabie et al., “Adaptive Optimal Control for Continuous-Time Linear Systems Based on Policy Iteration,” Automatica, vol. 45, no. 2, pp. 477-484, 2009.
[CrossRef] [Google Scholar] [Publisher Link]
[24] Richard S. Sutton, and Andrew G. Barto, Introduction to Reinforcement Learning, 1998.
[25] Kyriakos G. Vamvoudakis, “Online Learning Algorithms for Differential Dynamic Games and Optimal Control,” Federated Electronic Theses and Dissertation, pp. 1-218, 2011.
[Google Scholar] [Publisher Link]
[26] Yu Jiang, and Zhong-Ping Jiang, Robust Adaptive Dynamic Programming, Wiley, pp. 1-216, 2017.
[Google Scholar] [Publisher Link]
[27] J.J. Murray et al., “Adaptive Dynamic Programming,” IEEE Transactions on Systems, Man, and Cybernetics, Part C, vol. 32, no. 2, pp. 140-153, 2002.
[CrossRef] [Google Scholar] [Publisher Link]
[28] Yu Jiang, and Zhong-Ping Jiang, “Computational Adaptive Optimal Control for Continuous-Time Linear Systems with Completely Unknown Dynamics,” Automatica, vol. 48, no. 10, pp. 2699-2704, 2012.
[[CrossRef] [Google Scholar] [Publisher Link]
[29] Draguna Vrabie, and Frank Lewis, “Neural Network Approach to Continuous-Time Direct Adaptive Optimal Control for Partially Unknown Nonlinear Systems,” Neural Networks, vol. 22, no. 3, pp. 237-246, 2009.
[CrossRef] [Google Scholar] [Publisher Link]
[30] Kyriakos G. Vamvoudakis, and Frank L. Lewis, “Online Actor-Critic Algorithm to Solve the Continuous-Time Infinite Horizon Optimal Control Problem,” Automatica, vol. 46, no. 5, pp. 878-888, 2010.
[CrossRef] [Google Scholar] [Publisher Link]
[31] Yu Jiang, and Zhong-Ping Jiang, “Global Adaptive Dynamic Programming for Continuous-Time Nonlinear Systems,” IEEE Transactions on Automatic Control, vol. 60, no. 11, pp. 2917-2929, 2015.
[CrossRef] [Google Scholar] [Publisher Link]
[32] Quan-Yong Fan, and Guang-Hong Yang, “Adaptive Actor-Critic Design-Based Integral Sliding-Mode Control for Partially Unknown Nonlinear Systems with Input Disturbances,” IEEE Transactions on Neural Networks and Learning Systems, vol. 27, no. 1, pp. 165-177, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[33] Hoa Van Doan, and Nga Thi-Thuy Vu, “Adaptive Sliding Mode Control for Uncertain Wheel Mobile Robot,” International Journal of Electrical and Computer Engineering, vol. 13, no. 4, pp. 3939-3947, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[34] Kyriakos G. Vamvoudakis, and Frank L. Lewis, “Online Actor Critic Algorithm to Solve the Continuous-Time Infinite Horizon Optimal Control Problem,” International Joint Conference on Neural Networks, Atlanta, GA, USA, pp. 3180-3187, 2009.
[CrossRef] [Google Scholar] [Publisher Link]
[35] Murad Abu-Khalaf, and Frank L. Lewis, “Nearly Optimal Control Laws for Nonlinear Systems with Saturating Actuators Using a Neural Network HJB Approach,” Automatica, vol. 41, no. 5, pp. 779-791, 2005.
[CrossRef] [Google Scholar] [Publisher Link]
[36] Kyriakos G. Vamvoudakis, Draguna Vrabie, and Frank L. Lewis, “Online Adaptive Learning of Optimal Control Solutions Using Integral Reinforcement Learning,” 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), Paris, France, pp. 250-257, 2011.
[CrossRef] [Google Scholar] [Publisher Link]
[37] J. Yang, W.H. Chen, and S. Li, “Non-Linear Disturbance Observer-Based Robust Control for Systems with Mismatched Disturbances/Uncertainties,” IET Control Theory & Applications, vol. 5, no. 18, pp. 2053-2062, 2011.
[CrossRef] [Google Scholar] [Publisher Link]
[38] Keith Dupree et al., “Asymptotic Optimal Control of Uncertain Nonlinear Eulerlagrange Systems,” Automatica, vol. 47, no. 1, pp. 99107, 2010.
[CrossRef] [Google Scholar] [Publisher Link]