In this work, we propose the use of gradient-based optimization as a means for initializing an evolutionary algorithm population to approach deep reinforcement learning problems. We investigate the efficacy of this method on several standard reinforcement learning benchmarks by evaluating improvements in the policies of the resulting models and their sample complexity. We find that our hybrid approach surpasses DQN and GAs alone in common RL bechmarks, and show that a population of five agents per generation with good initialization could be used to outperform a population of five hundred randomly initialized agents.