Feature Reinforcement Learning Using Looping Suffix Trees.

There has recently been much interest in history-based methods using suffix trees to solve POMDPs. However, these suffix trees cannot efficiently represent environments that have long-term dependencies. We extend the recently introduced CTMDP algorithm to the space of looping suffix trees which have previously only been used in solving determinis- tic POMDPs. The resulting algorithm replicates results from CTMDP for environments with short term dependencies, while it outperforms LSTM-based methods on TMaze, a deep memory environment.