Deformable Geometry Based Semantic Reconstruction from Scene Graphs.

Structural scene graph based image generation provides a new paradigm for image-oriented semantic communications, whose goal is the semantic level rather than pixel-level reconstruction. The challenges include capturing relationships between objects and producing a reasonable geometric layout for each object accordingly. However, category information alone is not instructive enough for the generation process at the receiver side. Moreover, it is worth effort to extract the spatial dependencies among different objects in an image, therefore determine the object layouts on the whole instead of in an independent manner. In this paper, a deformable geometry framework for scene graph based image generation is proposed, in order to reconstruct images with higher semantic fidelity and visual pleasure. In particular, we introduce shape and appearance information to guide the generation process, from the scope of statistic modeling. Furthermore, we apply a spatial warping network to conduct geometric deformations on the layouts of different objects. Qualitative and quantitative experiments illustrate the superiority of our model compared to the state-of-the-art Sg2im method.