On the Audio Hallucinations in Large Audio-Video Language Models
CoRR(2024)
摘要
Large audio-video language models can generate descriptions for both video
and audio. However, they sometimes ignore audio content, producing audio
descriptions solely reliant on visual information. This paper refers to this as
audio hallucinations and analyzes them in large audio-video language models. We
gather 1,000 sentences by inquiring about audio information and annotate them
whether they contain hallucinations. If a sentence is hallucinated, we also
categorize the type of hallucination. The results reveal that 332 sentences are
hallucinated with distinct trends observed in nouns and verbs for each
hallucination type. Based on this, we tackle a task of audio hallucination
classification using pre-trained audio-text models in the zero-shot and
fine-tuning settings. Our experimental results reveal that the zero-shot models
achieve higher performance (52.2
fine-tuning models achieve 87.9
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要