Enabling blazing fast serving of LLMs. Early member of a core technical team building distributed inference runtime.

I’m the engineering lead (AI Framework area) for a company-wide initiative to enable AMD GPUs for Microsoft AI workloads. Architected and led the development for AMD backends of ONNXRuntime and in-house inference stack for OpenAI models. Enabled GPT4 models on MI300 GPUs with leading performance/price, announced in Microsoft Build. Worked across Microsoft teams to deploy the model for CoPilot applications.

Built a core AI Infra team. Architected and implemented a new ML framework from scratch with core features including hierarchical IR-based auto-differentiation, static operator scheduling and memory optimization, ML-guided optimizations for sparse computations. Led cross-team effort to improve the efficiency of company's GPU fleet and supported multiple product areas.

Google Brain/TensorFlow.

I'm a founding member of TensorFlow's graph optimization framework Grappler:(https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/grappler). I authored a few of Grappler's very first graph optimizations and later took much of the leadership role of the project. Helped grow the project from supporting only CPU and GPU to a variety of backends today: TensorFlow Lite, TensorFlow.js, TensorRT, nGraph, XLA, and more, and from zero graph optimizations to over a dozen targeting both training and inference for all accelerator backends.

Heterogeneous computer architecture and scientific computing