Building Ray (Ray Serve LLM, Ray Serve, Ray Data LLM) and vLLM!

Actively contributing to the open-source Ray project. Explore some of my contributions: Ray LLM (distributed inference), Ray Core (compiled graphs, NCCL).

Prime Video Personalization ML Platform team

1. Developed low-latency online inference microservices utilizing gRPC, TensorRT, and Triton on EKS with seamless MLOps CI/CD.

2. Architected distributed training cluster to scale out model training with Ray on EKS.

3. Built and scaled high-throughput data and ML processing pipelines with Spark, PyTorch, EMR, SageMaker, and Airflow.

4. Maintained scalable microservices based on AWS Fargate and ECS with 100k+ TPS.