Value Iteration on Multicore Processors

Value Iteration (VI) is a powerful, though time consuming, approach to solve reinforcement learning problems modeled as Markov Decision Processes (MDPs). In this paper, we explore strategies to run the sate-of-the-art cache efficient algorithm for VI developed by us [1], [2] on a multicore processor. We demonstrate a speedup of up to 2.59 on a 10-core multiprocessor using 20 threads on popular benchmark data. The speedup for the parallelized portion of the computation is up to 5.89.