Worked on production inference infrastructure for real-time financial ML models.

• Designed inference gateway serving 300–600 QPS (1000+ peak capacity) with adaptive batching, improving GPU utilization from ~40% to ~70%

• Reduced Triton model initialization latency from ~45s to ~8s via TensorRT engine caching and optimized loading strategies

• Implemented queue-depth-based autoscaling to stabilize performance during 2–3× traffic surges

• Built low-latency Redis feature cache (~90% hit rate, <5ms P95) for real-time model features

• Improved sustained throughput by ~60% through batch-size × concurrency tuning

• Supported a real-time recommendation engine for 100M+ DAUs, maintaining sub-4ms P99 latency under high concurrency.

• Built a distributed key-value store handling 1.5M QPS, boosting throughput by 38% and ensuring high availability.

• Designed zero-downtime rolling updates on Kubernetes, cutting peak memory usage by 50% and minimizing service disruption.

• Improved gRPC client performance with in-process caching and resilient retries, raising cache hit ratio by 65%.

• Deployed monitoring and automated recovery workflows with Prometheus/Grafana, reducing MTTR for SLO violations by 75%.

Mission: Architecting a cost-effective, production-grade GenAI platform for multimodal video understanding.

• Agentic Routing: Designed a latency-aware router orchestrating queries between Gemini and self-hosted vLLM (Llama 3); reduced operational costs by 40% without sacrificing quality.

• Automated Evaluation: Developed an LLM-as-a-judge pipeline to quantify groundedness and safety, reducing manual review cycles from days to minutes.

• Multimodal RAG: Built an event-driven pipeline leveraging TensorRT-optimized vision models for real-time video metadata generation and cataloging.

• Model Adaptation: Streamlined PEFT (LoRA) workflows, enabling 10x faster experimental iterations.

• Built a prototype music platform enabling users to upload tracks securely on a blockchain database (Estuary) to ensure immutability and copyright protection.

• Designed a music similarity detection algorithm analyzing rhythm, melody, and harmony to identify plagiarism risks and discover related tracks.

• Developed a user dashboard and music player with features for file management, playback, and similarity insights.

• Focused on scalability, algorithm accuracy, and security to deliver a reliable and privacy-first music sharing experience.

• DevOps & CI/CD: Migrated legacy monolith services to containerized Docker environments, reducing build-deploy times by 40%.

• Modernization: Refactored legacy codebases to TypeScript, significantly improving type safety and reducing runtime errors for production APIs.