Evaluating Collaborative Filtering Over Time

Collaborative Filtering (CF) evaluation centres on accuracy: researchers validate improvements over state of the art al- gorithms by showing that they reduce the mean error on predicted ratings. However, this evaluation method fails to reect the reality of deployed recommender systems, which operate algorithms that have to be iteratively updated as new users join the system and more ratings are input. In this work we outline a method for evaluating CF over time, and elaborate on work done exploring the temporal qualities of CF algorithms and recommendations.