Instruction finetuning of Multimodal Large Language Models (MLLMs) relies on massive datasets, hindering scalability and efficiency. Consequently, data pruning has emerged as a vital strategy to attain full-dataset performance using a minimal subset. However, existing methods often incur high computational overheads or suffer from performance collapse at high pruning rates. To address these challenges, we propose Decoupled Gradient-Guided Stratification (DGGS), a resource-efficient data pruning framework. Diverging from approaches that introduce additional computation via external proxy models, we natively repurpose the internal vision encoder to stratify data based on visual complexity, and employ the LLM module as a deterministic verifier to quantify the semantic density of each stratum. DGGS dynamically allocates the pruning budget to stratified regions where semantic importance outweighs visual complexity, filtering out visual noise. Empirically, DGGS operates with minimal additional memory overhead and achieves robust performance, retaining approximately 85% of full-dataset performance with 1% of the training data.
原文链接
分享
引用
