一种基于PSO的分层策略搜索算法<sup>*</sup>

Abstract
Figure/Table
References
Related Citation (3)

Download: PDF (403 KB) HTML (1 KB)
Export: BibTeX | EndNote (RIS)

Abstract In order to overcome drawbacks in hierarchical policy gradient reinforcement learning algorithm (HPGRL), such as problem of local optimum, a new algorithm for searching hierarchical policies is proposed, named Hierarchical Policy Search Based on PSO (PSOHPS). The designers create the task decomposition graph according to the hierarchical theory of MAXQ, one of the classical hierarchical reinforcement learning techniques. Then the hierarchical parameterized policies of all compound subtasks are evolved in process of direct interaction with the environment by utilizing a particle swarm to acquire the optimized action policies. Experimental results demonstrate the algorithm is valid and its performance outperforms that of HPGRL remarkably.

Key words： Hierarchical Reinforcement Learning Particle Swarm Optimization (PSO) Hierarchical Policies Negotiation Deadlock

Received: 07 December 2006

ZTFLH:

TP181

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	PENG ZhiPing
	LI ShaoPing

Cite this article:

PENG ZhiPing,LI ShaoPing. An Algorithm for Hierarchical Policy Search Based on PSO[J]. , 2008, 21(1): 98-103.

URL:

http://manu46.magtech.com.cn/Jweb_prai/EN/ OR http://manu46.magtech.com.cn/Jweb_prai/EN/Y2008/V21/I1/98

[1] Gao Yang, Chen Shifu, Lu Xin. Research on Reinforcement Learning Technology: A Review. Acta Automatica Sinica, 2004, 30(1): 86100 (in Chinese)
(高阳,陈世福,陆鑫.强化学习研究综述.自动化学报, 2004, 30(1): 86100)
[2] Barto A G, Mahadevan S. Recent Advances in Hierarchical Reinforcement Learning. Discrete Event Dynamic Systems: Theory and Applications, 2003, 13(4): 4177
[3] Li Wei, Ye Qingtai, Zhu Changming. Application of Hierarchical Reinforcement Learning in Engineering Domain. Journal of Systems Science and Systems Engineering, 2005, 14(2): 207217
[4] Puterman M. Markov Decision Processes. New York, USA: Wiley, 1994
[5] Su Chang, Gao Yang, Chen Shifu, et al. The Study of Recognizing Options Based on SMDP. Pattern Recognition and Artificial Intelligence, 2005, 18(6): 679684 (in Chinese)
(苏畅,高阳,陈世福,等.基于SMDP环境的自主生成options算法的研究.模式识别与人工智能, 2005, 18(6): 679684)
[6] Watkins C T, Dayan P. QLearning. Machine Learning, 1992, 8(3): 279292
[7] Baxter J, Bartlett P L. InfiniteHorizon PolicyGradient Estimation. Journal of Artificial Intelligence Research, 2001, 15(4): 319350
[8] Ghavamzadeh M. Hierarchical Reinforcement Learning in Continuous State and MultiAgent Environments. Ph.D Dissertation. Amherst, USA: University of Massachusetts. Graduate School, 2005
[9] Ghavamzadeh M, Mahadevan S. Hierarchical Policy Gradient Algorithms // Proc of the 20th International Conference on Machine Learning. Washington, USA, 2003: 226233
[10] Dietterich T G. Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition. Journal of Artificial Intelligence Research, 2000, 13(1): 227303
[11] Ghavamzadeh M, Mahadevan S, Makar R. Hierarchical Multiagent Reinforcement Learning. Journal of Autonomous Agents and MultiAgent Systems, 2006, 13(2): 197229
[12] Hu Xiaohui, Shi Yuhui, Eberhart R. Recent Advances in Particle Swarm // Proc of the IEEE Congress on Evolutionary Computation. Portland, USA, 2004, Ⅰ: 9097
[13] Peng Zhiping, Peng Hong, Zheng Qilun. Study on Bilateral and MultiIssue Autonomous Negotiation Model. Journal of Electronics & Information Technology, 2007, 29(3): 733738 (in Chinese)
(彭志平,彭宏,郑启伦.一种双边多议题自治协商模型的研究.电子与信息学报, 2007, 29(3): 733738)