Spatially modulated control of pattern formation in a general nonlocal nonlinear system

Spontaneous symmetry breaking and pattern formation are notable and significant natural phenomena that occur in nonlinear systems. Further advancing such research to Bose-Einstein condensates (BECs) is of much interest for both fundamental physics and practical applications. We investigate the formation and selection of diverse self-organized ground-state patterns, as well as the control of their structural phase transition in a general nonlocal nonlinear system described by nonlocal Gross-Pitaevskii equation (GPE). We show a homogeneous matter wave can undergo roton instability (RI) and their control is addressed by spatially weak modulated nonlinearity, which may result in the formation of various ordered patterns (including square, stripe, annular, and different hexagonal structures, etc.) with and without the trapping potential. In particular, our findings reveal that when a spatially weak modulated nonlinearity is introduced, a hexagonal ground-state pattern (which is the only type of structure observed without such modulation) can transform into several distinct pattern types. In contrast to previous research, the patterns identified in our study can be easily controlled by modifying the nonlinear parameters. Our work opens a pathway for versatile manipulation of self-organized patterns as well as the associated structural phase transitions.