My goal is to advance towards intelligent systems that help users express the shapes in motion they have in mind, and their organization intro complex virtual worlds. These users may be computer artists, but also the general public, engineers or scientists from other disciplines who have mental images of their objects of study, but cannot seamlessly communicate them. Our general methodology is to provide them expressive, gesture-based control (such as sketching, deformation, copy-pasting) while augmenting the graphical models they manipulate with knowledge, was it laws from priors or statistics and correlations learnt from examples.