基于路径积分强化学习方法的蛇形机器人目标导向运动

doi:10.16451/j.cnki.issn1003-6059.201901001

Abstract
Figure/Table
References
Related Citation (10)

Download: PDF (1653 KB) HTML (1 KB)
Export: BibTeX | EndNote (RIS)

Abstract Path integral is derived from stochastic optimal control. It is a numerical iteration method and solves the problem of the optimal control about continuous nonlinear systems at a high convergence speed without system model. A policy improvement algorithm based on path integral reinforcement learning is proposed for the target-directed locomotion of a snake-like robot in this paper. The path integral reinforcement learning approach is employed to learn the parameters of the snake-like robot serpentine equation, and the robot is controlled to arrive at the target position fast without contacting obstacles in simulation environment. Moreover, the robot with the priori knowledge from the simulation in real environment can complete the task well. Experimental result verifies the validity of the propose algorithm.

Received: 10 September 2018

ZTFLH:

TP 242.6

Fund:Supported by National Natural Science Foundation of China(No.61603200,U1613210)

Corresponding Authors: FANG Yongchun , Ph.D., professor. His research interests include visual control of robots, quadrotor, underactuated crane system and micro/nano manipulation.

About author:: ZHU Wei, master student. His research interests include snake-like robot and deep reinforcement learning.GUO Xian, Ph.D., lecturer. His research interests include snake-like robot and deep reinforcement learning.

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	FANG Yongchun
	ZHU Wei
	GUO Xian

Cite this article:

FANG Yongchun,ZHU Wei,GUO Xian. Target-Directed Locomotion of a Snake-Like Robot Based on Path Integral Reinforcement Learning[J]. , 2019, 32(1): 1-9.

URL:

http://manu46.magtech.com.cn/Jweb_prai/EN/10.16451/j.cnki.issn1003-6059.201901001 OR http://manu46.magtech.com.cn/Jweb_prai/EN/Y2019/V32/I1/1

[1] HIROSE B S. Biologically Inspired Robots: Snake-Like Locomotors and Manipulators. Oxford, UK: Oxford University Press, 1993.
[2] KELASIDI E, LILJEBACK P, PETTERSEN K Y, et al. Innovation in Underwater Robots: Biologically Inspired Swimming Snake Robots. IEEE Robotics and Automation Magazine, 2016, 23(1): 44-62.
[3] BORENSTEIN J, HANSEN M, BORRELL A. The OmniTread OT-4 Serpentine Robot-Design and Performance. Journal of Field Robo-tics, 2007, 24(7): 601-621.
[4] ROLLINSON D, CHOSET H. Pipe Network Locomotion with a Snake Robot. Journal of Field Robotics, 2014, 33(3): 322-336.
[5] TANAKA M, NAKAJIMA M, SUZUKI Y, et al. Development and Control of Articulated Mobile Robot for Climbing Steep Stairs. IEEE/ASME Transactions on Mechatronics, 2018, 23(2): 531-541.
[6] SATO M, FUKAYA M, IWASAKI T. Serpentine Locomotion with Robotic Snakes. IEEE Control Systems Magazine, 2002, 22(1): 64-81.
[7] ROLLINSON D, CHOSET H. Gait-Based Compliant Control for Snake Robots // Proc of the IEEE International Conference on Robotics and Automation. Washington, USA: IEEE, 2013: 5123-5128.
[8] WU X D, MA S G. Adaptive Creeping Locomotion of a CPG-Controlled Snake-Like Robot to Environment Change. Autonomous Robots, 2010, 28(3): 283-294.
[9] CRESPI A, IJSPEERT A J. Online Optimization of Swimming and Crawling in an Amphibious Snake Robot. IEEE Transactions on Robotics, 2008, 24(1): 75-87.
[10] MATSUNO F, MOGI K. Redundancy Controllable System and Control of Snake Robots Based on Kinematic Model // Proc of the IEEE Conference on Decision and Control. Washington, USA: IEEE, 2000, V: 4791-4796.
[11] MOHAMMADI A, REZAPOUR E, MAGGIORE M, et al. Maneuvering Control of Planar Snake Robots Using Virtual Holonomic Constraints. IEEE Transactions on Control Systems Technology, 2015, 24(3): 884-899.
[12] ARIIZUMI R, MATSUNO F. Dynamic Analysis of Three Snake Robot Gaits. IEEE Transactions on Robotics, 2017, 33(5): 1075-1087.
[13] OKAL B, ARRAS K O. Learning Socially Normative Robot Navigation Behaviors with Bayesian Inverse Reinforcement Learning // Proc of the IEEE International Conference on Robotics and Automation. Washington, USA: IEEE, 2016: 2889-2895.
[14] KRETZSCHMAR H, SPIES M, SPRUNK C, et al. Socially Compliant Mobile Robot Navigation via Inverse Reinforcement Lear-ning. International Journal of Robotics Research, 2016, 35(11): 1289-1307.
[15] ZHU Y K, MOTTAGHI R, KOLVE E, et al. Target-Driven Visual Navigation in Indoor Scenes Using Deep Reinforcement Learning // Proc of the IEEE International Conference on Robotics and Automation. Washington, USA: IEEE, 2017: 3357-3364.
[16] GONG C H, TRAVERS M J, ASTLEY H C, et al. Kinematic Gait Synthesis for Snake Robots. International Journal of Robotics Research, 2016, 35(1/2/3): 100-113.
[17] THEODOROU E, BUCHLI J, SCHAAL S. A Generalized Path Integral Control Approach to Reinforcement Learning. Journal of Machine Learning Research, 2010, 11: 3137-3181.
[18] WILLIAMS G, DREWS P, GOLDFAIN B, et al. Aggressive Dri-ving with Model Predictive Path Integral Control // Proc of the IEEE International Conference on Robotics and Automation. Washington, USA: IEEE, 2016: 1433-1440.
[19] CHEBOTAR Y, KALAKRISHNAN M, YAHYA A, et al. Path Integral Guided Policy Search[J/OL]. [2018-08-23]. https://arxiv.org/pdf/1610.00529.pdf.
[20] OKADA M, RIGAZIO L, AOSHIMA T. Path Integral Networks: End-to-End Differentiable Optimal Control[J/OL]. [2018-08-23]. https://arxiv.org/pdf/1706.09597.pdf.
[21] CHATTERJEE S, NACHSTEDT T, WORGOTTER F, et al. Reinforcement Learning Approach to Generate Goal-Directed Locomotion of a Snake-Like Robot with Screw-Drive Units // Proc of the 23rd International Conference on Robotics in Alpe-Adria-Danube Region. Washington, USA: IEEE, 2014. DOI: 10.1109/RAAD.2014.7002234.
[22] POREZ M, IJSPEERT A J. Improved Lighthill Fish Swimming Mo-del for Bio-inspired Robots: Modeling, Computational Aspects and Experimental Comparisons. The International Journal of Robotics Research, 2014, 33(10): 1322-1341.