Referring image segmentation with attention guided cross modal fusion for semantic oriented languages

1 Introduction Referring image segmentation aims to identify and segment the instance by language description.It is a challenge task due to the combination of both language and visual domains[1].The current mainstream methods mainly focus on English,and can achieve an appropriated performance.However,it cannot achieve the similar performance while they are directly applied to semantic-oriented languages like Chinese due to the different language characteristics.English is structure-centra-lized because English sentences are always well-formatted by articles,auxiliary verb,conj unction and preposition.Semantic-oriented languages like Chinese[2],Japanese and Thai are relatively volatile which seldom use auxiliary word,and they are often unsegmented languages,so it brings more challenge for algorithm to identify key words and analyze the meaning.Additionally,the lack of the collaborative learning of word-level and multimodal-level attention between visual and linguistic modalities makes it more difficult for current approaches to adapt to semantic-oriented languages.