Mafin: Enhancing Black-Box Embeddings with Model Augmented Fine-Tuning
CoRR(2024)
摘要
Retrieval Augmented Generation (RAG) has emerged as an effective solution for
mitigating hallucinations in Large Language Models (LLMs). The retrieval stage
in RAG typically involves a pre-trained embedding model, which converts queries
and passages into vectors to capture their semantics. However, a standard
pre-trained embedding model may exhibit sub-optimal performance when applied to
specific domain knowledge, necessitating fine-tuning. This paper addresses
scenarios where the embeddings are only available from a black-box model. We
introduce Model augmented fine-tuning (Mafin) – a novel approach for
fine-tuning a black-box embedding model by augmenting it with a trainable
embedding model. Our results demonstrate that Mafin significantly enhances the
performance of the black-box embeddings by only requiring the training of a
small augmented model. We validate the effectiveness of our method on both
labeled and unlabeled datasets, illustrating its broad applicability and
efficiency.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要