Policy Evaluation Using the \Omega -Return

We propose the Omega-return as an alternative to the Omega-return currently used by the TD (lambda) family of algorithms. The benefit of the Omega-return is that it accounts for the correlation of different length returns. Because it is difficult to compute exactly, we suggest one way of approximating the Omega-return. We provide empirical studies that suggest that it is superior to the lambda-return and gamma-return for a variety of problems.