A long standing goal in artificial intelligence has been to develop rational agents that can reliably interact with the environment and perform interesting tasks. While traditional control theory enables us to efficiently develop agents for environments whose dynamics can be modelled with reasonable accuracy, in many real world problems the dynamics of the environment is unknown to the agent due to the inherent complexity of the environment. A common approach is to model this problem as a Markov Decision Process (MDP). In reinforcement learning (RL) algorithms, the agent explores the environment to gain knowledge about it while simultaneously trying to maximize the accumulated reward it receives. Large scale RL problems with continuous state-action spaces pose several challenges. They require generalizing the experience obtained by the agent by tedious feature selection for function approximation. For real-time applications, the RL algorithm must also ensure efficient action selection. Lastly, they must be computationally efficient. This project investigates and extends a recent algorithm designed to address this issue. Non-parametric approximate linear programming (NP-ALP) is a sample-based approach to value function approximation for MDPs with continuous stateand action-spaces, which requires only a distance function and an estimate of the Lipschitz constant of the value function. Using these inputs and a set of training data, NP-ALP finds a value function that is consistent with the specified Lipschitz constant. Previous theoretical work provided appealing sample
分享
引用
