Reinforcement Learning Contextual Bandits

This note is an introduction to contextual bandits, a class of multi-armed bandits where an agent must take sequential actions at time t = 1, 2, . . . based on observed rewards that are supposed to depend on some unknown parameter θ (the context). In recommander systems for instance, the parameter θ is supposed to characterize the user. This parameter is learnt based on the feedback provided by the user for each proposed item. Each item corresponds to an action of the agent, who must learn the best actions, i.e., the items providing the best rewards. We mainly focus on so-called linear bandits, where expected rewards are linear functions of the actions, then present the extension to logistic bandits.