Medical visual question answering (VQA) is a key medical AI challenge, but scarce data limits progress. Current methods prioritize more pre-training data, overlooking medical data's inherent constraints (ethics, privacy, specialization) which cause slower accumulation of data than public web data. Sole reliance on more data risks rapid performance plateaus. To address these challenges, this paper proposes the cross-modality discriminative pattern identification model (CMDPI), which fundamentally rethinks training methodologies to provide a data-efficient framework for medical VQA. During pre-training, CMDPI identifies inherent biases in conventional techniques under conditions of data scarcity and introduces a co-regularization approach that integrates multiple pretraining techniques for regularization to enhance model generalizability. For fine-tuning, a difference reconstruction mechanism is proposed that effectively preserves unique discriminative features, and head mixup is introduced to further remedy the issue of overfitting. Experimental results demonstrate that CMDPI achieves performance comparable to or surpassing existing methods while requiring substantially less pre-training data. Our work shows the viability of optimizing training paradigms rather than pursuing indiscriminate data scaling for advancing medical VQA systems.
原文链接
分享
引用
