My research focuses on creating better and more general language models. To do so, I’ve had the chance to work with amazing collaborators on:

(a) large language models: Developing data-constrained scaling laws, building the largest open models (BLOOMZ/mT0 & BLOOM) & large code models (OctoPack, StarCoder & SantaCoder)

(b) language embeddings: Building the largest embedding benchmark (MTEB) & state-of-the-art retrieval models (GRIT & SGPT)

(c) multimodal models: Won 2nd/3300 in Meta AI’s Hateful Memes Challenge (Blog, Paper)