TINA: Think, Interaction, and Action Framework for Zero-Shot Vision Language Navigation
arxiv(2024)
摘要
Zero-shot navigation is a critical challenge in Vision-Language Navigation
(VLN) tasks, where the ability to adapt to unfamiliar instructions and to act
in unknown environments is essential. Existing supervised learning-based
models, trained using annotated data through reinforcement learning, exhibit
limitations in generalization capabilities. Large Language Models (LLMs), with
their extensive knowledge and emergent reasoning abilities, present a potential
pathway for achieving zero-shot navigation. This paper presents a VLN agent
based on LLMs, exploring approaches to the zero-shot navigation problem. To
compensate for the shortcomings of LLMs in environmental perception, we propose
the Thinking, Interacting, and Action (TINA) framework. TINA enables the agent
to scrutinize perceptual information and autonomously query key clues within
the environment through an introduced question-answering module, thereby
aligning instructions with specific perceptual data. The navigation agent's
perceptual abilities are enhanced through the TINA framework, while the
explicit thought and query processes also improve the navigational procedure's
explainability and transparency. We evaluate the performance of our method on
the Room-to-Room dataset. The experiment results indicate that our approach
improves the navigation performance of LLM-based agents. Our approach also
outperformed some supervised learning-based methods, highlighting its efficacy
in zero-shot navigation.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要