My broad research goal is reducing the capability gap between compact language models (LMs) and extreme scale frontier models. I have worked with Yejin Choi to design learning and inference algorithms to unlock capabilities in smaller models, for example:
I introduced Symbolic Knowledge Distillation, a framework to capture the knowledge of large LMs in compact expert models. With coauthors, I have applied this to summarization, dialogue, generic knowledge, pluralistic values, and even multi-modal knowledge with computer vision. This line recieved an outstanding paper award at EMNLP 2023
I have worked to develop a suite of decoding-time methods to empower compact and efficient LMs, including BottleSum, Reflective Decoding, NeuroLogic Decoding, and the DeLorean technique. This line recieved a best method paper award at NAACL 2022
In parallel, I study LM capabilities and their limits, particularly how frontier models diverge from human expectations. For example, I proposed the Generative AI Paradox to question the relative abilities to understand and generate in these models