Rethinking Model Prototyping through the MedMNIST+ Dataset Collection
CoRR(2024)
摘要
The integration of deep learning based systems in clinical practice is often
impeded by challenges rooted in limited and heterogeneous medical datasets. In
addition, prioritization of marginal performance improvements on a few,
narrowly scoped benchmarks over clinical applicability has slowed down
meaningful algorithmic progress. This trend often results in excessive
fine-tuning of existing methods to achieve state-of-the-art performance on
selected datasets rather than fostering clinically relevant innovations. In
response, this work presents a comprehensive benchmark for the MedMNIST+
database to diversify the evaluation landscape and conduct a thorough analysis
of common convolutional neural networks (CNNs) and Transformer-based
architectures, for medical image classification. Our evaluation encompasses
various medical datasets, training methodologies, and input resolutions, aiming
to reassess the strengths and limitations of widely used model variants. Our
findings suggest that computationally efficient training schemes and modern
foundation models hold promise in bridging the gap between expensive end-to-end
training and more resource-refined approaches. Additionally, contrary to
prevailing assumptions, we observe that higher resolutions may not consistently
improve performance beyond a certain threshold, advocating for the use of lower
resolutions, particularly in prototyping stages, to expedite processing.
Notably, our analysis reaffirms the competitiveness of convolutional models
compared to ViT-based architectures emphasizing the importance of comprehending
the intrinsic capabilities of different model architectures. Moreover, we hope
that our standardized evaluation framework will help enhance transparency,
reproducibility, and comparability on the MedMNIST+ dataset collection as well
as future research within the field. Code will be released soon.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要