Which One? Leveraging Context Between Objects and Multiple Views for Language Grounding
arxiv(2023)
Abstract
When connecting objects and their language referents in an embodied 3D
environment, it is important to note that: (1) an object can be better
characterized by leveraging comparative information between itself and other
objects, and (2) an object's appearance can vary with camera position. As such,
we present the Multi-view Approach to Grounding in Context (MAGiC), which
selects an object referent based on language that distinguishes between two
similar objects. By pragmatically reasoning over both objects and across
multiple views of those objects, MAGiC improves over the state-of-the-art model
on the SNARE object reference task with a relative error reduction of 12.9%
(representing an absolute improvement of 2.7%). Ablation studies show that
reasoning jointly over object referent candidates and multiple views of each
object both contribute to improved accuracy. Code:
https://github.com/rcorona/magic_snare/
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined