Direct reciprocity between individuals that use different strategy spaces

In repeated interactions, players can use strategies that respond to the outcome of previous rounds. Much of the existing literature on direct reciprocity assumes that all competing individuals use the same strategy space. Here, we study both learning and evolutionary dynamics of players that differ in the strategy space they explore. We focus on the infinitely repeated donation game and compare three natural strategy spaces: memory-1 strategies, which consider the last moves of both players, reactive strategies, which respond to the last move of the co-player, and unconditional strategies. These three strategy spaces differ in the memory capacity that is needed. We compute the long term average payoff that is achieved in a pairwise learning process. We find that smaller strategy spaces can dominate larger ones. For weak selection, unconditional players dominate both reactive and memory-1 players. For intermediate selection, reactive players dominate memory-1 players. Only for strong selection and low cost-to-benefit ratio, memory-1 players dominate the others. We observe that the supergame between strategy spaces can be a social dilemma: maximum payoff is achieved if both players explore a larger strategy space, but smaller strategy spaces dominate. Author summary Direct reciprocity can lead to cooperation between individuals who meet in repeated encounters. The shadow of the future casts an incentive to cooperate. If I cooperate today, you may cooperate tomorrow. But if I defect today, you may defect tomorrow. In most studies of direct reciprocity it is assumed that both players explore the same space of possible strategies. In contrast, here we study interactions between players that use different strategy spaces and therefore utilize different memory capacities. Surprisingly, we find that more complex strategy spaces often lose out against simpler ones. The social optimum, however, is achieved if all players use the more complex space. Therefore, the game between strategy spaces becomes a higher order social dilemma.