My research focuses on building more reliable and adaptable foundation models in realistic scenarios using data-centric methods:
Exploring Pre-Training Data Imperfections: Investigate the effects of different types of data imperfections (e.g., corruption, bias, diversity) during pre-training and how they influence the physics of foundation models.
Understanding and Mitigating Catastrophic Inheritance: Examine how imperfections in pre-training data impact downstream tasks, and develop fine-tuning methods to mitigate the detrimental effects, thereby improving reliability and generalization.
Leveraging Imperfect Data and Labels for Transfer Learning: Develop robust learning methods to handle various types of imperfect data and labels for effective downstream task adaptation.
Understanding Multi-Modality Generalization from Individual Components: Advancing multi-modality foundation models by understanding their individual components from training data.