Real-Time Monte Carlo Tree Search in Ms Pac-Man
Computational Intelligence and AI in Games, IEEE Transactions (2014)
摘要
In this paper, Monte Carlo tree search (MCTS) is introduced for controlling the Pac-Man character in the real-time game Ms Pac-Man. MCTS is used to find an optimal path for an agent at each turn, determining the move to make based on the results of numerous randomized simulations. Several enhancements are introduced in order to adapt MCTS to the real-time domain. Ms Pac-Man is an arcade game, in which the protagonist has several goals but no conclusive terminal state. Unlike games such as Chess or Go there is no state in which the player wins the game. Instead, the game has two subgoals, 1) surviving and 2) scoring as many points as possible. Decisions must be made in a strict time constraint of 40 ms. The Pac-Man agent has to compete with a range of different ghost teams, hence limited assumptions can be made about their behavior. In order to expand the capabilities of existing MCTS agents, four enhancements are discussed: 1) a variable-depth tree; 2) simulation strategies for the ghost team and Pac-Man; 3) including long-term goals in scoring; and 4) reusing the search tree for several moves with a decay factor γ. The agent described in this paper was entered in both the 2012 World Congress on Computational Intelligence (WCCI'12, Brisbane, Qld., Australia) and the 2012 IEEE Conference on Computational Intelligence and Games (CIG'12, Granada, Spain) Pac-Man Versus Ghost Team competitions, where it achieved second and first places, respectively. In the experiments, we show that using MCTS is a viable technique for the Pac-Man agent. Moreover, the enhancements improve overall performance against four different ghost teams.
更多查看译文
关键词
Monte Carlo methods,computer games,digital simulation,tree searching,2012 IEEE Conference on Computational Intelligence and Games,2012 World Congress on Computational Intelligence,MCTS,Ms Pac-Man,Pac-Man agent,Pac-Man-Ghost Team competitions,arcade game,real-time Monte Carlo tree search,scoring,simulation strategies,surviving,time 40 ms,variable-depth tree,Monte Carlo,Monte Carlo tree search (MCTS),Pac-Man,real time
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络