Set up tools and infrastructure to allow spinning up new models in hours

Designed and generated Python and Typescript SDK to get models in the hands of customers within the day of the request

Optimized various model inference with batching, vLLM and PyTorch, reducing latency of pipelines by up to 80%

Led the setup of tracing by integrating OpenTelemetry with Jaeger and augmented Ray logs and Prometheus metrics to allow SDK developers to get features out in hours rather than days and help pinpoint bottlenecks in customers’ AI pipelines

Architected and led the implementation of the Control Plane-Data Plane split utilizing Kubernetes Operator pattern to make the system declarative, resulting in the creation of the egress only Data Plane agents to be installed in customers’ VPC that are orchestrated by the hosted Control Plane, leading to significant reduction of deployment time from weeks to just hours

add observability to the CLI by integrating Segment Analytics to track user actions

using the observability metrics and customer's feedback to improve user experience

add automatic refreshing of authentication token, and support for JSON and YAML output to allow scripting of the CLI

Implemented automated CLI binaries build, testing and release pipeline involving Goreleaser tool, bash script, GNU Make script, AWS S3 and Azure DevOps Pipelines

Automated end-to-end cloud system test for Cloud CLI

Taught data structures using Java