Stochastic replicator dynamics and evolutionary stability

To develop the concept of evolutionary stability in a stochastic environment, we investigate the continuoustime dynamics of a two-phenotype linear evolutionary game with generally correlated random payoffs in pairwise interactions. By using the Gram-Schmidt orthogonalization procedure and Ito's formula, we deduce a stochastic differential equation for the phenotype frequencies that extends the replicator equation, called the stochastic replicator equation. We give conditions for stochastic stability of a fixation state or a constant interior equilibrium point with respect to the stochastic dynamics of the two phenotypes. We show that, if a fixation state is stochastically stable, then the pure strategy corresponding to this fixation state must be stochastically evolutionarily stable with respect to mixed strategies. However, this is not the case for a mixed strategy that corresponds to a stochastically stable constant interior equilibrium point with respect to the two phenotypes.