ALF: Adaptive Label Finetuning for Scene Graph Generation
arxiv(2023)
摘要
Scene Graph Generation (SGG) endeavors to predict the relationships between
subjects and objects in a given image. Nevertheless, the long-tail distribution
of relations often leads to biased prediction on coarse labels, presenting a
substantial hurdle in SGG. To address this issue, researchers focus on unbiased
SGG and introduce data transfer methods to transfer coarse-grained predicates
into fine-grained ones across the entire dataset. However, these methods
encounter two primary challenges: 1) They overlook the inherent context
constraints imposed by subject-object pairs, leading to erroneous relations
transfer. 2) Additional retraining process are required after the data
transfer, which incurs substantial computational costs. To overcome these
limitations, we introduce the first plug-and-play one-stage data transfer
pipeline in SGG, termed Adaptive Label Finetuning (ALF), which eliminates the
need for extra retraining sessions and meanwhile significantly enhance models'
relation recognition capability across various SGG benchmark approaches.
Specifically, ALF consists of two components: Adaptive Label Construction (ALC)
and Adaptive Iterative Learning (AIL). By imposing Predicate-Context
Constraints within relation space, ALC adaptively re-ranks and selects
candidate relations in reference to model's predictive logits utilizing the
Restriction-Based Judgment techniques, achieving robust relation transfer.
Supervised with labels transferred by ALC, AIL iteratively finetunes the SGG
models in an auto-regressive manner, which mitigates the substantial
computational costs arising from the retraining process. Extensive experiments
demonstrate that ALF achieves a 16
typical SGG method Motif, with only a 6
to the state-of-the-art method IETrans.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要