Platform & data storage for LangSmith AI Observability

* Engineer on the the big data orchestration team. The team built and managed

Netflix's big data orchestration platform, Maestro, used by thousands of data scientists, data engineers, and ML engineers to power their workflows.

* Rearchitect the data orchestration platform to a distributed and scalable system, Maestro, built

on Spring Boot, Conductor and CockroachDB.

* Lead subprojects with partners including the ML/Metaflow team and Titus compute. This included architecture, development, and alignment with the various partners.

* Area-lead for the distributed systems design interview track. Overhauled the interview process for the data organization.

* Team lead and founding engineer on the Data Workflow Management team. The team grew and managed the data orchestration platform at Uber which was used company-wide (3k+ users).

* Re-architected the data orchestration platform into a multi-tenant, highly available, distributed system, capable of operating at Uber’s scale.

* Enhanced engineering brand of the team: gave conference talks including Strata Data Conference, SF Big Analytics Meetup, also authored an Uber engineering blog on the team's work .

* Engineering lead for data-wide programs for continuously improving reliability of data systems. Established reliability best practices across data teams, influenced/enabled teams to effectively adopt these, resulting in significantly improved infrastructure stability.

* Drove technical direction and vision of the teams products, while working in a cross-functional manner with management, PMs, and stakeholders to set team's architecture, roadmap and OKRs.

* Implemented features and functionality for SpinalQ, a multithreaded JRuby application responsible for the company's ElasticSearch data ingestion pipeline.

* Overhauled the company's ElasticSearch cluster for fault tolerance and performance through cluster configuration, query selection, monitoring, alerting, and replication setup.

* Developed functionality for Command Center, a communication platform used by facility staff built on top of JRuby, Rails, and React.js.

* Environment: JRuby / ElasticSearch / Rails / JavaScript / React.js / MySQL