DFATRAN: Beyond Static Features for Dynamic Transferability Estimation

Transferability estimation is pivotal for the judicious selection of appropriate pre-trained models in the context of downstream target tasks, given that fine-tuning all candidate models is computationally prohibitive. Recent studies primarily center on evaluating the discriminative power of static features. However, these approaches often fail to effectively capture the dynamic evolution of model representations throughout the fine-tuning process. To address this limitation, we propose a novel and effective method dubbed DFATran (Dynamic Feature Adjustment Transferability Estimation). DFATran leverages an adversarial feature perturbation mechanism to generate perturbations specifically targeting the decision boundary, thereby effectively simulating the dynamic characteristics of the fine-tuning process. Concurrently, DFA-Tran is further designed to mitigate the correlation within intra-class features and suppress potential spurious noise, fostering features that better conform to the characteristics of adaptive representations. Extensive experimental results unequivocally demonstrate the superior performance of our proposed method, achieving an increase of up to 20.72% in the average τw across common benchmarks.