Learning to swim efficiently in a nonuniform flow field

Microswimmers can acquire information on the surrounding fluid by sensing mechanical queues. They can then navigate in response to these signals. We analyze this navigation by combining deep reinforcement learning with direct numerical simulations to resolve the hydrodynamics. We study how local and nonlocal information can be used to train a swimmer to achieve particular swimming tasks in a nonuniform flow field, in particular, a zigzag shear flow. The swimming tasks are (1) learning how to swim in the vorticity direction, (2) learning how to swim in the shear-gradient direction, and (3) learning how to swim in the shear-flow direction. We find that access to laboratory frame information on the swimmer's instantaneous orientation is all that is required in order to reach the optimal policy for tasks (1) and (2). However, information on both the translational and rotational velocities seems to be required to accomplish task (3). Inspired by biological microorganisms, we also consider the case where the swimmers sense local information, i.e., surface hydrodynamic forces, together with a signal direction. This might correspond to gravity or, for microorganisms with light sensors, a light source. In this case, we show that the swimmer can reach a comparable level of performance to that of a swimmer with access to laboratory frame variables. We also analyze the role of different swimming modes, i.e., pusher, puller, and neutral.