ML Infra/MLOps & Backend - $8m in funding, backed by Daniel Gross, Felicis Ventures, etc.
•Led end-to-end integration of large language models that powers core AI functionalities encompassing backend development, ML infrastructure, MLOps(model fine-tuning, serving/deployment, evaluation, and data wrangling & pipeline)
•Developed core backend components in Go, including ML service integration, A/B testing, content-based recommendation system, improving engagement and data-driven optimizations utilizing Postgres, ClickHouse, Elasticsearch
•Architected and maintained distributed GPU machine learning infrastructure using Kubernetes and Grafana, scaling to handle 1 million daily requests with 98% uptime while ensuring high availability, fault tolerance, and observability
•Cut infrastructure costs by $400K/yr through LLM inference optimization using concurrency tuning, batching, quantization
•Built data pipeline for extensive data wrangling of 15M+ data points and fine-tuned Large Language Models with curated data, increasing user satisfaction by 25% with Pandas, WandB, Axolotl