Interests: I’m interested in computer systems for large-scale workloads such as AI, data analytics and cloud computing. My recent projects include programming models for LLM applications, efficient runtimes for ML and analytics, quality assurance tools and AI-based data analytics systems. I am also interested in data privacy, and have worked on systems that can provide scalable privacy for communication, Internet queries and SaaS applications.
Open Source: Most of my research work is open source. During my PhD, I started the Apache Spark project, which is now one of the most widely used frameworks for distributed data processing, and co-started other datacenter software such as Apache Mesos and Spark Streaming. At Stanford, we developed DAWNBench, a machine learning performance competition that drew submissions from the top industry groups and influenced the industry-standard MLPerf, and we are developing a wide range of open source software including Weld, NoScope, FlexFlow, ColBERT and DSP. I was also involved in the Databricks project to develop Dolly, the first fully commercially usable, open source instruction-following LLM, and its open source instruction-tuning dataset.